Defense Wins Championships - Part 1
by MJesser on Apr 13, 2008 4:03 AM EDT
15 comments
We have all heard basketball and football coaches preach that "defense wins championships". Does the same hold true for baseball?
As an organization, the Twins have been among the leaders in voicing their opinion about team defense. Terms such as "Small Ball" and "Doing the little things" have become second nature to most Twins fans. These terms have served us well over the past 6 - 7 years. It is easy to look back and say that if the Twins have "done the little things right" by playing solid defense (among other things), and they have won their division 4 out of the last 6 years, then clearly things like playing solid defense is an important component of making the playoffs.
We have a lot of ground to cover, so let's jump into the data:
The first statistic I used to track defense was errors, partly because it is a simple continuous statisic and partly because it's easy to interpret. As with all these graphs, please note that the World Series data is much too small to make any real inferences about. I debated if I should pull it out of the graphs, but ultimately decided it was still fun to look at, and also because these graphs start to look a little bare when only plotting two lines.
The first take-away from this graph is that that there is a fairly consistent gap between playoff teams and non-playoff teams. The average difference between the two was 7.5, meaning that the average playoff team made 7.5 less errors during the regular season than the average non-playoff team.
Stat Geek Warning: I ran a t-test for two groups and found the difference between groups is significant at p>.01
The other major take-away from this graph is the fairly steady down-trend of errors over the past 8 years. My gut tells me that this is due to the post-steroid era and the increasing emphasis on speed, defense and, as I mentioned above, "doing the little things". However, there may be other factors at work, such as recent changes to rules (I just heard on a Cubs broadcast about an obscure rule change regarding errors made in foul territory). Another consideration is that errors are subjective. The downward trend could represent a decrease in the scorekeeper's willingness to charge errors.
The next statistic I used to track defense was Bill James' Defensive Efficiency. I honestly don't know the details about how the statisic is calculated, but here is a brief summary from www.baseball-reference.com:
This is a Bill James measure that estimates the number of batted balls turned into outs by a team. The estimate for plays made is based on outs minus things like double plays, caught stealings and outfield assists or total batters faced minus strikeouts, walks, HBP, Hits and errors times a factor.
The only take-away I have from this graph is that the difference between Playoff and non-playoff teams has decreased, but is still present
Stat Geek Warning: The t-test for two groups is still significant at p>.01
Now that we have adequately researched what the data looks like, I'd like to turn our attention to a more rigorous statistical approach. We're moving beyond Correlation and into Causation. It's all great and good that lower error rates are correlated with playoff appearances, but what we are really trying to uncover is if having a good defense will Predict playoff appearances. For this, we turn to the statistical method, multiple regression.
Quick Stats Disclaimer: I understand that not everyone can understand complex statistical procedures and have tried to be as friendly to non-stats people as possible. I have highlighted the most important numbers but will not attempt to give a full explanation of how to read the chart.
If your eyes start to glaze over, skip to the highlighted portions of the last two boxes as they will be the easiest to understand. If you really, really hate stats, just skip to the graphical chart below.
This chart attempts to take my defensive metrics and show how well they predict regular season wins.
The model shows that Defensive Efficiency is a slightly stronger predictor of wins than is errors. The last box, entitled "Predictors" allows you to calculate what it would take, according to this model, to gain 1 win. For example, say a team had 110 errors at the end of the season. If the team had reduced their errors by 5 (rounded from 5.48), we could predict that they would have won 1 more game.
Now, to put this graphically, I have charted wins along the bottom and the standardized defensive metrics along the left (I standardized the scores so that I could put both errors and DefEff on the same graph). A zero on the defensive metrics can be interpreted as average.
As you can see, there is a fairly linear relationship between defensive metrics and wins. As wins increase, your DefEff increases and your Errors decrease.
Next up: Defense Metrics as Predictors of Playoff Appearances
1 recs |
15 comments
|
Email
Print
Comments
fun
nice work. I know your disclaimers were not for me personally, but I'll respond. This stuff is great. I don't hate stats, just can't do 'em. The graphs are great, the charts are meaningless (to me).
Thanks again for distilling some of the info down to a couple of easily understood summary sentences.
by montanatwinsfan on Apr 13, 2008 10:26 AM EDT reply actions
Great work
I like what you've done. Did you buy a software package, or are you doing this in excel?
I really like that you tested wins related to defense, instead of defense by year, with discrete buckets by no-playoffs/playoffs/WS. This is definitely the best way to analyze it.
I apologize in advance for the following critique/suggestion, because I know I haven't put in the time to do any of these analyses yet, but here goes:
I think that your analysis would be greatly improved by throwing out the graphs that show data by year. I think that the "better" way, is to find an on-level factor, to bring all years to the same level. So, for errors, look at league-wide error rates, calculate the ratio of errors in any year to errors in 2007, and bring all statistics to "2007 level". Same for defensive effeciency, etc. Then, your graphs can compare the "causation" statistics (errors) on the x-axis, and the outcome (wins) on the y-axis. I also thing that on-level statistics will improve the credibility of your data, which will probably reduce your parameter error, resulting in lower variance and improved test statistics (higher adjusted R-squared).
By separating data in this way, it may also provide a credible enough data set to compare the relative contributions to wins of OBP/OPS/Defense/ERA/etc. The other interesting thing it would afford is an analysis of the correlation between these different statistics. Do the teams with better offenses have better/worse/independent defenses.
Anyway, I don't mean to criticize. I really like the analysis, and I think this is what separates a blog from those that just optimistic forecasts from a bunch of homers.
Excel
Actually, everything is done in Excel 2007. The charts are done by linking charts to pivot tables on the raw data.
There's also a "stats pack" add on for Excel. The regression option creates tables basically like the one above. It's greate for running simple regressions, but is pretty limited in its options.
What would my life be like without the '91 World Series?
OK
Nice data. I realize it's too tempting (and more fun) to draw conclusions from the data, but the conclusions should really have additional data behind them, or they're just guesses. Case in point: the idea that the drop in errors by year since 2001 might be a post-steroid adjustment. Based on just the data presented, it's possible to argue that there are more steroids in the game now than before -- after all, outfielders make fewer errors per play than infielders, and nobody can make an error on a home run. If more fly balls are being hit by year, and more fly balls means more steroids, then...you get the idea.
A few more observations:
- The greater variability of the 'WS' teams is likely due to their far smaller sample size: there are 30 teams in MLB, and 8 that make the post-season each year, but only two get to the WS. The data could also provide support for Billy Beane's lament that all a good GM and manager can do is get a team to the playoffs; once there, it's more about luck and who gets hot than raw skill.
- The negative correlation between DPs and wins might seem counterintuitive -- aren't DPs a good thing? -- but I'd guess if more data were run that the trend you note would hold up. After all, we have other data that suggests that the largest factor in total team DPs isn't how good the team is at turning DPs (rate of successful DPs per opportunity), but rather how many baserunners that team allows (more opportunities). We also have very well-established data on the relationships between runs and wins, and the relationships between baserunners and runs. It's speculation, but I think it would be supported if more numbers were run -- if you're leading the league in DP effieciency, you're probably doing OK, but if you're leading the league in DP opportunities, you probably have a crap pitching staff. (Note that the same argument should apply in reverse -- if you lead the league in offensive DPs because you also lead the league in OBP, that's not such a bad deal.)
- Your last sentence should probably be reversed: it's not that your defense gets better as you win more games; you win more games because your defense gets better.
On the whole, though, this is interesting work. I'll look forward to seeing more.
Dont be too tough on him for the simplistic summaries
I, for one, have asked him to distill some of this info down into summary statements for people, (like me) who don't know stats. I worked very hard to avoid any statistics classes in grad school becuase numbers are really hard for me. Therefore, I have requested he put some of these ideas into words for me at the risk of being called out for oversimplifying.
by montanatwinsfan on Apr 13, 2008 11:32 AM EDT up reply actions
Thanks
I'm always looking for feedback, so I appreciate your critique. Really, this is my first attempt at using these more advanced statistics to get meaningful data outside of a classroom setting.
Let me respond to your points:
1. I wasn't really trying to "conclude" that the post-steroid era was the reason for the drop in errors, and quite honestly, it wasn't the point of the post. I was simply making a "guess" as to why it may be going down (based on the conventional wisdom that teams are focusing more on speed and defense) , and was sure to provide other possible reasons to explain.
2. I mentioned several times in my last post on payroll/playoffs that the WS data was basically pointless, and attempted to remind everyone again why there's nothing we can do with the data. Despite that I included it because I thought it would at least be interesting, even if conclusions aren't made on the data.
2. THANK YOU for your thoughts on the DP metric. I made sure to avoid it as much as possible because I simply couldn't come up with a way to explain it. I think this makes a lot of sense. Perhaps we'll have to dive into that in a future post.
3. You're right, the predictors should be on the x-axis and the dependent should be on the Y-axis. I tried and the graph didn't look very nice. I went with this this instead. Really, the argument can be made that this isn't really showing regression any more than the first two graphs. It's a different way to look at correlation. I haven't been able to come up with a really good way to graph regressions. If you have any thoughts, I'd love to hear them.
What would my life be like without the '91 World Series?
the charts are interesting
but there is nothing here to suggest that defense causes more winning. you have shown that they are mildly correlated with that regression, but since the regression does not include any of the biggest predictors of wins (team ERA, OPS, HR, K, etc) your point estimates suffer from omitted variable bias. Given the low R^2 (so defense does not explain much) and the omission of all of the important variables, it is impossible to draw any conclusions from your estimates. For instance, the conclusion that five fewer errors holding other factors constant would lead to 1 more win is incorrect. If it were true, the average non-playoff team would have only had to commit 10 fewer errors in order to make the playoffs (as this is the average difference you report initially). The fact that this statement is so wrong should have been a red flag about your estimates.
My guess (and it is just a guess) is that when you include all the relevant variables that explain wins, some or all of your defensive metrics will become insignificant. It is also possible, as I believe snolls mentioned, that good defenses are highly correlated with good pitching or good hitting. In this case there could be other difficulties with the regression, but it is important to find out.
Laslty (and this may betray my own ignorance as I have never had a good stats class) I am not sure what is meant by "standardized coefficients". Well, I understand it, but I am wondering how you standardized them. When I read work like this, that is the sort of thing I want to know. If there is a "standard" way of creating standardized coefficients (sorry for the tautology), then I guess I just don't know it.
http://noblingblings.blogspot.com/
More Predictors
It's a fine line to walk when adding predictors to the regression model. Down the road, I plan on running a larger regression that includes RS, RA, Hits, Walks, HR, HR Allowed, but distilling that information into one post would be too much.
I've optioned for taking little bits at a time and working my way up to that model. In the meantime I wanted to take a look at certain "aspects" of the game. This also allows me to isolate and compare a statistic like errors against a "newer" statistic such as DefEff.
What would my life be like without the '91 World Series?
Backwards
You have also read my "Predictors" chart backwards. 1 win = -5.48 errors.
If it were true, the average non-playoff team would have only had to commit 10 fewer errors in order to make the playoffs
10 fewer errors, according the data I have provided would result in approx. 2 more wins, and certainly would not guarantee a playoff spot.
What would my life be like without the '91 World Series?
I'll reply to both in one post
I didn't interpret anything backwards, although I admit that I might not be interpreting everything correctly because I don't know what your model is. You said the average difference in errors between playoff and nonplayoff teams was 10 errors (early in your post). The whole idea of a regression is to isolate the effects of a single variable (say errors), holding other factors constant.
I admit that I don't actually know what your regression is because you didn't write it in the post, but I am assuming the model is wins = a + b(defensive metrics) + e. I also admit that I don't know how you came up with your predictors. However, however you did it the prediction that your modelgives is that 10 fewer errors yields two more wins. How do you reconcile this with the fact that the difference on average between playoff and non-playoff teams is only 10 errors?
The way I reconcile it is that, since your model does not include offensive or pitching statistics, the coefficients on the defensive metrics are picking up any correlation between defense and these other variables, which are probably quite high. So the coefficient on errors is not actually measuring the effect of errors. It is measuring the effect of errors and all the omitted variables that are correlated with errors. Thus, you can't trust the point estimate, or even really the sign on the point estimate.
I don't mean to be overly critical, but this is important stuff, especially when you are presenting it to people who don't completely understand your methods. You can't "isolate and compare a statistic like errors against a "newer" statistic such as DefEff" with a miss-specified model, because all your inference is biased.
What I find really interesting about your results is that DefEff, errors and DPs are all sufficiently uncorrelated with each other that you don't have any multicollinearity issues in the regression. This is certainly telling about how sophisticated some of our measures of defensive ability really are!
http://noblingblings.blogspot.com/
two more things
1) i read your profile and it actually describes me word for word. my most vivid early baseball memory is the 91 world series, and i think we were born in the same year. the combination of losing kirby puckett and the strike chased me from baseball for a few years, but i found my way back in college.
2) I agree that dwintheiser makes a really good point about DPs. And I think this illustrates the point that I am trying to make in my comments: you can't really find out the effect of double plays unless you account for the number of baserunners a team allows. For instance, holding the number of baserunners constant, DPs should add to wins. But in order to measure this, you need to incorporate number of baserunners into the regression. I am not even sure where you find data like that.
http://noblingblings.blogspot.com/
Standardized Coefficients
If you really must know, the formula for standardized coefficients is:
Beta / [(St Dev of X) / (St Dev of Y)]
What would my life be like without the '91 World Series?
So, I'm trying to follow
Are you saying that teams that commit fewer errors win more games?
If so, I might have been able to deduce that with fewer steps.
If you are saying that the team that commits the fewest errors wins the World Series x percent of the time, that might draw some interest.
Though mainly from official MLB scorers, who might think they can get their team to the World Series simply by awarding more hits than errors.
Yesterday in Boston, Cano hit a routine chopper to 2B, Pedroia fielded and, in transferring the ball to make a totally routine but quick throw, he dropped it. Base hit Cano.
Both players were happy with that, as were their agents.
I think there has to be some unreliable data in using total errors, including: pitching quality, SO rates, WHIP rates, pitches thrown, closeness of games, field quality, weather--there are a myriad of variables that impact that number.
Also, you play Giambi at 1B and everyone in the infield will get more errors. But, he might just win you the game with his bat...
Conclusions
Being a two part post on defensive metrics, I'm not trying to say anything... yet.
Part 1 was written to lay a ground work for my next article, which will show how little defensive stats play into the grand scheme of things. Ultimately, I plan to show that while playoff teams typically have better defenses (graph 1 and graph 2), a better defense cannot actually pridict a playoff spot.
I'll admit though, I probably should have done a better job explaining this in my first post.
What would my life be like without the '91 World Series?
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Recommended FanPosts
Recent FanPosts
In Association With
Big League Stew - MLB - Yahoo! Sports
- The Juice: Cry havoc, R.A. Dickey, and let slip the dancin' knuckler
- Jeff Passan talks trade deadline, A-Rod's 600th and Dr. Claw
- Create-a-Caption: Buck Showalter ponders the challenge ahead
- Finally: Roy Oswalt to the Phillies is a done deal
- Rangers still owe A-Rod, Mickey Tettleton and others some cash
Certain photos copyright © 2010 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
