Credibility 101

by snolls on Jun 18, 2009 12:58 PM EDT Comment 53 comments

Lets say you are in lower manhattan, NYC, and you need to get to LaGuardia for a flight in 30 minutes. On average, cab drivers in NY make it to LaGuardia in less than 30 minutes 27.5% of the time. Drivers percentages vary from 22.5% to 32.5%.

You have two cab drivers in front of you. Driver (A) has been driving for 10 years (5000 trips), and makes the trip in 30 minutes 30% of the time. Driver (B) just started driving, has done the trip 5 times, and made it in time twice. Which cab should you take?

The answer, plus a basic lesson in statistics, after the break.

The answer is that you should take the driver who has performed better than average for 10 years. The odds are that driver B got lucky, and will not continue to demonstrate that success. Notice that this example can be turned into batting averages, give or take. If Carlos Gomez gets two hits in his first two at bats, does that make him a career .400 hitter? Yes, for now. Do you expect him to hit at .400 for the rest of his career?

Terms like small sample size, regress toward to the mean get thrown around here a lot (I'm really just quoting Ubelmann). But, at this site, and more so around the internet/blogosphere/tv/newspaper (ahem Hartman!), people don't really seem to understand it. I tried to find a good link to an explanation of credibilty, but failed, so I'll draw up the "lesson plan" myself. I hope this makes sense, I'm not much of a teacher.

Alright, lets say you are flipping a fair coin (50% heads). If you flip the coin once, you will get either 1 (heads) or 0 (tails). If you get a 1, do you expect to always get heads in the future? But, there is a 50% chance after 1 flip that you will end up with a "career" average of 1. If you flip it twice, there is a 25% chance that you will have a career average of 1, even though the "true" average is .5. If you flip it 10 times, there is a 0.1% chance that you still have a career average of 1. If you flip it 10 times, there is an 38% chance that you have a career average of at least .6. However, if you go to 100 flips, the probability of having a career average of at least .6 drops to 3%.

Hopefully this makes sense so far. The point being, that flipping a coin is a random process, and sometimes fair coins will show averages that aren't equal to the true probability. As the amount of experience rises (flips), the probability of remaining just as far away from the true mean decreases. Remember that for all of the cases mentioned above, no matter how many flips and how many heads, you would always have a 50% chance of heads in future flips.

Lets make it a bit more complicated. You have two coins, one is fair (50% heads), and one is not (60% heads). You have flipped a randomly chosen coin 10 times, and you have more than 6 heads heads. What are the odds of getting a heads on your next flip? This depends on the probability that your coin is the fair, or the weighted coin. Well, the fair coin had a 38% chance, and the weighted coin has a 63% chance of getting at least 6 heads. Since you randomly chose your coin, there is a 62% chance (63/(63+38)) that the coin you have chosen is the weighted coin. Your future EXPECTED flip is 62%*60% + 38%*50% = 56%. Note that this is lower than the average flip thusfar (which is greater than or equal to .6). In fact, it is very close to the mean flip of 55% (average of 50% and 60%).

In reality there is a different set of odds for 6 heads, 7 heads, 8 heads, but using "cumulative" distributions made my job easier. I will probably have my math corrected anyhow, because I'm racing to do this and get back to my work.

The idea is that this extends to any baseball player. Joe Mauer has a career average of something like .320. There is a chance that he is actually a .320 hitter, and has performed exactly as expected, that he is actually a .400 hitter, and has been unlucky, or that he is actually a .200 hitter, and should be expected to regress. The world of people hitting .320 is made of of mostly players who truly hit worse than .320 hitters, some who truly are .320 hitters, and a few that are truly better than .320 hitters. The odds are that any .320 hitter has gotten lucky, to some extent. The fact that he has performed above average does not make him a .320 hitter, but it does increase the probability that he is an above average hitter. However, the odds are less than 50% that he is truly a .320 hitter, because the average hitter is worse than .320.

I hope this makes sense. I'd love to see a good discussion about this, because I think that a lot of smart baseball people on this board don't completely understand the logic behind this. The other factors, like age, injuries, coaching, experience, etc complicate this model, but all have to be considered in aggregate to get fair, accurate projections.

0 recs | Comment 53 comments

Story-email Email Printer Print

Comments

Very well explained

Much better than my relatively poor attempts, though I did make the point similarly the Mauer thread eventually.

This was what I was trying to get to—more guys who hit any particular number are actually closer on “true talent” the the average than they are further away. Thus we make the assumption that any particular player is closer to average on true talent, even though it’s possible we are wrong in some instances.

There’s another, somewhat related point I’d like to make regarding luck and randomness in baseball. Take clutch hitting, a skill about which there is considerable debate as to whether it exists. I tend to think it doesn’t, but I’ve been wrong before. Here’s the point: even if you could show me a group of players that consistently hit well in the clutch, and another group that consistently didn’t, that wouldn’t go very far in proving that clutch ability exists.

Why? The coin flip thing. Most people, if you ask them to construct a series of 100 coin flip results without actually flipping a coin—in other words, just make it up, won’t include enough long “streaks” of heads or tails. An expert could tell the difference between actual results and made up results, even if they had the same totals at the end.

The point is that even if clutch hitting is completely random, you would expect to find some guys who do well in this area in 9 years out of 10, and some who do poorly in 9 years out of 10.

Anyway, thanks for this cogent explanation of something I was trying to say but lacked the complete understanding in order to do so.

by Eric in Madison on Jun 18, 2009 1:31 PM EDT reply actions 0 recs

Different point I think... but still a worthy point...

Well, your point in the other thread than the point here. The league average is incorporated into projection models as a smoothing affect. To account for degradation of play due to injuries and such. Tango describes it here:

http://www.hardballtimes.com/main/article/forecasting-2006/

The regression step means that projects “appear” low for great players. No one will be projected to hit more than 30 HR’s or 112 RBI’s when we all know that league leaders always go well over those two marks. But many players will get hurt or have bad years and you can’t predict who those players will be going into the season.

In my opinion, this trick really only makes sense for above average players. I mean, would you regress Matt Tolbert to the league average? I suppose this works because when above-average players degrade they keep playing, but when below average players degrade, they get replaced.

by DavidRF on Jun 18, 2009 1:57 PM EDT up reply actions 0 recs

To vs toward

Not to the league average, but “toward” the league average. Again, I’m not saying some players aren’t better than others. Just that of all players with above average statistics, on average, they have been lucky. Not every player with above average statistics, but on average.

by snolls on Jun 18, 2009 6:22 PM EDT up reply actions 0 recs

IMO, nobody regresses toward the league average...

Instead, each player will regress up or down toward their own “true” average, i.e., the average we would expect given an infinite number of at bats. And this personal average differs for each player. It just averages, across all players, to the league average.

For teh sake of argument, in the case of Mauer, suppose his “true” average at this point in his development / career is .335. For Tolbert, suppose his “true” average is .240. Each player will have a tendency to regress, regardless of whether thay are above or below the league average.

by Adam Peterson on Jun 18, 2009 10:07 PM EDT up reply actions 0 recs

yup

and, since we don’t know what their “true” average is, our best estimate should be somewhere between their own actual performance, and league average, with relative weight dependent on the “credibility” (aka sample size) of their performance.

by snolls on Jun 19, 2009 1:41 PM EDT up reply actions 0 recs

I understand the probability

But players aren’t coins. Their improvement and regression is not random. For example, good players tend to play better as they age to the point of 28 years old and worse as they age past 28 years old. This is not random. It’s a general principal that was proven by Bill James. And it is not the only feature of improvement and regression that is not random—injuries, e.g. The best projection systems take these facts into account.

My critique of the probabilistic views on Joe Mauer’s likely regression is the sense that regression is not only inevitable, but not governed by anything but random chance. I gave reasons why I think his regression is not random, which were mostly greeted with “Homer!” dismissals.

Yes I’m a homer. But I’m more interested in being right than being a homer. If there was a good reason to believe that Joe Mauer would regress below his career average beyond random chance, I would buy it. Given his age and experience, there is better reason to suggest that he would regress to somewhere above his career average than that he would regress below it. All but one person dismissed Bill James’ science in favor of randomness.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 18, 2009 3:42 PM EDT reply actions 0 recs

and for me,

Cmath’s explanation – in plain English – is much easier to understand.

by montanatwinsfan on Jun 18, 2009 3:49 PM EDT up reply actions 0 recs

Its both... its noisy data

Certainly a players development and improvement is not random. But there’s a lot of noise in baseball statistics. In any game, the best player can go 0-4, or the worst scrub can go 3-4. Players go on hitting streaks where every dribbler finds a whole and go into slumps where every hard-hit ball is right at someone.

Baseball’s not like basketball where you can just pass it to the star in the final seconds and judge him on the outcome… you never know who is going to be up for critical moments and even when the star is up, mighty casey just might strike out.

Players even have “career years”, so often it takes more than a season to estimate a players "base talent level. That’s the big monkey-wrench in all these projections. By the time you find the signal beneath the noise, the signal has changed.

And it works both ways, so its not a “homer” thing. Often a guy gets called up from the minors, goes 5 for 25, and there’s a push to weigh that tiny sample in the majors more than years of star-play in the minors and then they give up on him. Justin goes through 5-25 stretches all the time, but we know that’s not the real Justin.

So, players aren’t coins, but they are really more like “stacked decks”. Being a great player is important in the long run, but in the short run, it also helps to get good cards.

by DavidRF on Jun 18, 2009 4:51 PM EDT up reply actions 0 recs

Noisy data

Yes baseball data is noisy. But being a fan is about seeing a signal in the noise. I don’t watch baseball to appreciate the noise. I watch it to appreciate the signal If I wanted to watch randomness in action, I’d go play the penny slots.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 18, 2009 8:38 PM EDT up reply actions 0 recs

Proof

I think I generally agree with the points you are trying to raise, cmathewson, but I would question the use of the word ‘prove’ in regards to peaking at age 28 and would offer the substitute word ‘observation’, in that what Bill James observed in the data is a general increase in ability leading up to age 28 and a general decrease in ability (or production I guess) after age 28. Who knows how prevalent steroids were and how much they’ll skew the data for awhile, but the thing with statistics is that they are best at revealing broad truths about a given data set, yet (honestly) pretty insufficient for predicting a result on a ‘flip by flip’ basis.

I also think that it is quite simplistic to expect every major league hitter to regress to the MLB mean—and I don’t mean that in a disparaging way at all. I just think that the degree of variability in the data sample (all MLB hitters with their different approaches, goals, physical abilities, talent, coaching/mentoring, etc etc) precludes gleaning useful meaning (or predictions) from regressing hitters as different as Mauer and Gomez to the MLB mean. Some guys just have better phyiscal gifts than others, and some learn better approaches than others, and that should count for something in statistical analysis, just like the statistical analysis should be able to factor in that, given 1000 AB’s, even I will be able to x number of balls in play against major league pitching. The trouble, as pointed out, is that by the time you get enough samples to find the signal, the signal has most likely already changed.

"Come on Eddie, let's get serious."

by biggity2bit on Jun 19, 2009 12:11 AM EDT up reply actions 0 recs

Point taken

The thing about James is, he studied data for 80 years of major league baseball, and the bell curve was a pattern that emerged. So it’s not technically a proof. You really can’t prove something like that. But it’s pretty credible considering the data quantity. Oh, and his first study was done before wide-spread steroid use. It would be interesting to see how it changed in the 80s and 90s. My sense is that for every Bonds there was a Boone.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 19, 2009 12:15 AM EDT up reply actions 0 recs

26-28

That’s what I’ve read. Its not a rule, its a trend. Some players peak earlier, some later. Batting average might peak a bit earlier with power a little later and plate discipline perhaps later still. Stolen bases peak much earlier.

by DavidRF on Jun 19, 2009 2:26 AM EDT up reply actions 0 recs

Re: Appreciating the noise...

Well, we’re certainly appreciating the noise right now! Not even Joe’s mom thinks he’s a .425 hitter. Its big fun seeing that big number though.

You are right, all this noise analysis and probability estimates is indeed a bit geeky, but there’s a lot of time between his at bats. :-) Still a lot of season left… its like those probability cones on hurricane path projections. Hopefully we’ll still be doing this in three months and we can see that cone narrowing. :-)

by DavidRF on Jun 19, 2009 2:36 AM EDT up reply actions 0 recs

Indeed

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 19, 2009 9:09 AM EDT up reply actions 0 recs

... "somewhere above"

I’m certainly willing to admit that at age 26, he’s still getting better as a player. We could just be clashing on the details. Right now, he’s batting 78 points above his career high. None of Bill James’ models would project a spike that sharp.

Then there’s the issue of his previous baseline…. two batting titles in three years. Exactly how much better can Mauer get? Prince Fielder hit 50 homers at age 23, that doesn’t project him to 70 homers by age 27. I mean, that would be possible, but no one would ever project that.

The point is that its very difficult to predict a superlative. All you can say is “this guys probably going to be one of the best players in baseball” and then enjoy the show. Makes you appreciate the superlatives more when/if they happen if you understand how difficult it is to achieve them.

by DavidRF on Jun 18, 2009 5:01 PM EDT up reply actions 0 recs

Yeah

I was mostly reacting to people who think that it’s inevitable that he’ll hit .310 the rest of the way just because the statistical models suggest regression to the mean, where the mean is an average baseball player. It makes no sense to me to suggest that Joe Mauer will regress to average. He could, of course. Anything’s possible. But it is not at all likely given his career so far.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 18, 2009 6:30 PM EDT up reply actions 0 recs

Players are coins

At least when it comes to our ability to break down the cause/effect, events in baseball are part of a random process.

Players are just really complicated coins. Their true probability (how fair vs weighted they are) changes due to practice, learning, body development, injuries, coaching, etc. It is still a random process, and the rules of random events still apply. Thus, a player with a career average of .400 probably isn’t truly a .400 hitter.

Lets say that of all .320 career hitters at age 26, on average one would expect that they were “true” .300 hitters, which is still better than average. If we also expect the average player to improve by .10 points in their 27 and 28 years, then you would EXPECT Mauer to produce .310 for the next two years. Unless you have other information that should be considered.

All I’m saying is that we should adjust for the likelihood of luck, just as we adjust for the likelihood of injury, of “smart” baseball players learning, of bodies getting stronger, etc.

So, my question for you is, what makes you think that Mauer is a .340-.381 hitter (or whatever numbers you used), other than the fact that you have observed him to hit that well in his 2.5 years of leading the league. If that is your support, than I think you are over-projecting. However, if you are making specific adjustments for other factors, I’d be very curious to read how you reasoned it out.

by snolls on Jun 18, 2009 6:28 PM EDT up reply actions 0 recs

I think Adam has it right

Look at the standard deviation from his career average during healthy seasons (remember, he played hurt during his off years), factor in expected improvement given the standard age/performance bell curve, and consider that he has already hit .426 over 43 games with 94 to play. You get an average of around .340 the rest of the way and .380 overall.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 18, 2009 6:39 PM EDT up reply actions 0 recs

playing hurt...

Isn’t he hurt this season? I mean he’s already spent 22 games on the DL this season. I mean it certainly hasn’t slowed him down so far, but who knows as the season moves on. I certainly hope I’m wrong but it seems unreasonable to throw out his two .295 seasons because “that won’t happen again”.

I said in the other thread that anywhere in the .320-.340 range is a reasonable guess.

by DavidRF on Jun 19, 2009 2:16 AM EDT up reply actions 0 recs

By playing hurt

I mean he played through parts of two seasons with active injuries that affected his results. I don’t count his injury right now as active. He has it under control and it’s not affecting his results.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 19, 2009 9:11 AM EDT up reply actions 0 recs

Two things he's added to his game

*Power: Based on his current home-run pace relative to his career home run pace, he should get more hits per at bat than he has in the past. In his past rate, he would get about 7 homers in 350 at bats. At this rate, he should get 26 more homers in 350 at bats. That’s 19 more hits (which would have been warning track fly outs) than his career average going into the year. Add 19 hits to his career average coming into this year (.319) over his career average of at bats (500) and you get 38 additional points of batting average over his career average.

*Pulling the ball: If you look at Mauer’s spray charts this year versus previous years, you will see that he still hits the ball up the middle and to left field a lot. But he’s hitting the ball to the right side more this year—about three more times per week. I don’t know how to quantify the affects of this on his BA, but if you use the whole field, it is harder to pitch you and defend you and you should get more hits.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 18, 2009 7:47 PM EDT up reply actions 0 recs

really

You expect him to hit almost 40 HR this year? HR are a random process, just like hits. Isn’t it possible that part of the increase in his home run rate is a true change, and part of it is luck? After his first at-bat of the season, did you think that he had “added” power, and would hit 400 HR this year, or that he just happened to hit a HR, which he does from time to time, in his first AB of the season.

by snolls on Jun 19, 2009 1:45 PM EDT up reply actions 0 recs

Home runs are not luck

The principle for xFIP works for pitchers as well as hitters: There are three things that are not influenced by luck in baseball: walks, strike outs, and home runs. I can buy luck for BABIP outside of homers. But if you hit a homer, it was not a lucky hit.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 19, 2009 7:29 PM EDT up reply actions 0 recs

Disagree

Again, what I’m trying to say is that all events in baseball are “lucky” in that they are the result of a stochastic (random) process. Where they not luck, then A-Rod would hit a HR every 12th at bat (or whatever the number it is), like clockwork. But that’s not the way it works, the player has to guess/read the pitch properly, time the swing right, etc.

Basically, all I’m saying is this: Joe Mauer hitting 13 home runs in his first 1/3rd of the year does not mean that he will hit 39 for the season. It is a random process, and there is always a chance that people have in the past, and will in the future, perform differently than their “true expectation”.

by snolls on Jun 22, 2009 2:24 PM EDT up reply actions 0 recs

Complexity and randomness

I think you’re confusing complexity with randomness. Though you can’t predict exactly how a complex, nonlinear system will behave (“like clockwork”), you can identify patterns and, given enough data and computational power, you can map out typical scenarios and assign probability to each. But you don’t stop looking at patterns and developing computational models because of complexity.

The more variables you have, the more complex the system. And, at a certain point, a system will be too complex to compute. Even if you can develop a computer simulation of a system, the best supercomputers would take centuries to run the simulation. For all intents and purposes, these systems are random.

But a lot of stuff you are calling random has a perfectly reasonable, though complex, computational model. The more data you have to fill in your variables, the better your computational model. Baseball has a century of data that make its computational models pretty accurate. According to the model, walks, strikeouts, and homers are not random, but are governed by a relatively orderly, though complex, process.

I’m not saying Joe Mauer will hit 40 homers this year. But he’s on track to hit 40 if he continues at this pace. You asked if he added anything to his game to make me believe he will do better this year than last. I said power, which, if he continues at this pace, would account for a 40 point increase in batting average this year over last. I didn’t arrive at that number through spinning a roulette wheel. I used a time-tested computational model.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 22, 2009 3:13 PM EDT up reply actions 0 recs

Complexity

You are right. This is a very complex process. Chaos theory at its best.

I’ll avoid getting into whether things are predetermined or random, but either way that are incaclulable, so for our purposes, random. I’m not saying you spun a roulette wheel, I’m saying that your time tested computational model is the same as as the guy at roulette wheel who sees that two of the last 10 spins have been 00’s, and recognizes a trend. Because of the small sample size, your HR projection is just as unreliable. Why not use the other gambling logic, and assume that the streak can’t continue, and bet that he’ll won’t hit any more HR this year?

I’m not saying skill isn’t involved, but I don’t agree with your “time-tested computational model”. Hits are not more lucky than HR, there are just more variables in play. Players spend a lot of time trying to pull or push the ball, keep it fair, hit it on the ground in the infield, or on the fly to the outfield, etc. Defenders and pitchers try to control this aspect of the game as well.

A time tested computational model would look like this: Take all players in history with a similar number of at-bats as Mauer had at the beginning of the season. (You can decide how many other factors need to be identical). Then compare their career HR rates to their home run rates through the first 200 ABs (or whatever we are at). Use a linear model, with two variables (the two HR rates), to figure out which is the stronger contributor to HR rates over the rest of the year.

by snolls on Jun 23, 2009 9:21 AM EDT up reply actions 0 recs

Cause and effect

Maybe I should put it differently. Unlike roulette, hitting and pitching are governed by cause and effect. A guy adds strength and starts hitting balls over the fence that had been warning-track fly balls in previous years. Those homers are not random events but are caused, in part, by his increased strength.

The developer of FIP and xFIP examined 100 years of data to avoid small sample sizes. The model is therefore not subject to your argument. If you spin a roulette wheel enough times, eventually every number will come up an equal number of times. That’s the kind of sample sizes sabertatrics people use to develop their models, to control for complexity. Based on that model, home runs are not random occurrences.

If that is what you require of a computational model, you’ll never get it with Mauer. Mauer is unlike any player in history.

You see the world as a noisy series of random events, which can only be explained by probabilities. I reject your Humean skepticism and choose to see the world as a loosely correlated set of orderly, yet complex, systems. I doubt we will come to a middle ground on that one.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 23, 2009 10:09 AM EDT up reply actions 0 recs

Not necessarily

If you spin a roulette wheel enough times, eventually every number will come up an equal number of times.

This strikes me as a bit of a gambler’s fallacy here – there’s no reason to believe every number will come out equal in the end. There’s an equal chance of every number coming up on each spin, so even if 00 comes up twice in a row, there’s still no better chance the next time that it will land on something else. The odds favor a fairly even distribution throughout all the numbers, but they almost certainly won’t actually be equal, and there likely will be quite a few outliers.

Interestingly, if you’re asked to choose which of two series of coin flips (e.g. “heads, tails, heads, heads, tails”) is random and which was made up by a person, it’s almost always the case that the more random-looking series was the human-created one – a person will rarely put more than three or four of the same flip in a row, since it would seem too non-random, whereas chance has no such qualms.

"There are only two things that are infinite, the universe and human stupidity, and I'm not sure about the former." - Albert Einstein

by BeefMaster on Jun 23, 2009 11:45 AM EDT up reply actions 0 recs

Not the gambler's falacy

The probability of one number coming up more often than the rest approaches zero as the number of spins approaches infinity. I don’t know how many spins it takes to get to the point where the probability is minimal (or zero for all intents and purposes—I’m sure snolls knows), but it’s a big number. That is not the gambler’s fallacy. That’s a mathematical theorem.

The number of plate appearances in the history of baseball is also a big number. For all intents and purposes, the sheer quantity of data supporting the model normalizes for the manifestations of complexity that make baseball appear like a series of random acts.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 23, 2009 12:33 PM EDT up reply actions 0 recs

Mauer

If Mauer is unlike any other player in history, then how can your projections be any more meaningful than mine, which are based on the theory that what we see is “a noisy series of random events”.
I don’t see how your data, which is based on a similar premise regarding historical data from 100 years of games (players historically get better in certain age brackets, adding strength improves HR, more HR improves batting average), but less robust mathematical application.

by snolls on Jun 23, 2009 3:39 PM EDT up reply actions 0 recs

Or put in simpler terms...

Mauer’s “true” average would have increased over the past few years toward his peak.

The problem is, each player would also have a “true” peak year as well. I would expect this to differ from player to player, perhaps based on college versus high school, north versus south, position, etc.

by Adam Peterson on Jun 18, 2009 10:09 PM EDT up reply actions 0 recs

Yes

The bell curve is itself an average. For every Jim Kaat there’s a Dave Boswell. For every Rod Carew, there’s a Chuck KnoblauchAnd there are outliers, of course. It would be interesting to see how it differs between high school, college and international players. My sense is high schoolers and international players tend to peak earlier than college players, and burn out a little earlier as well.

In Mauer’s case, I think he’s right on the bell curve. He just got an early start in the majors because of starting out of high school and skipping levels. But his body is just coming into its prime this year. He finally stopped growing vertically and he’s put about 10 pounds of muscle on his core with all the back exercises. But he’s not at his peak yet. He’ll be pretty good in a couple of years.

"You're thinking too much. Just have fun." -- Bennie "The Jet" Rodriguez in Sandlot

by cmathewson on Jun 18, 2009 10:47 PM EDT up reply actions 0 recs

Speaking as a mathematically challenged individual...

paragraph 3 (after the break) is relatively understandable and not too painful. Paragraph 5 is a complete waste of time. That is not meant in any way to be a criticism, just a friendly reminder that if you hope to draw from a wider audience, or have people who are mathematically challenged – like myself – pay attention or understand, you have to keep things WAY simpler than paragraph 5. You lost me at the first sentence when I knew I would not be able to manage the numbers for two different coins that have separate percentages. There was not much point in reading any further.

Therefore, to remain inclusive – or to avoid frustrated ire from technophobic troglodytes- make sure you present a summary of any complicated statistical analyses with a relatively straightforward summary in english – no numbers. If that type of summary is less than perfect to describe your mathematical hypothesis, just make sure you point that out so that other statheads don’t jump all over your simplification.

by montanatwinsfan on Jun 18, 2009 3:47 PM EDT reply actions 0 recs

Agreed

It is often hard, when you spend a lot of time specializing in something, not wanting to be very technical and specific. Trying to put things in simple terms, and ignoring the counterexamples is difficult.

What I am trying to say, in as simple terms as possible (the rest of the post is just supporting examples), is this:
-——————————————-
When statistics differ from large sample averages, one should expect PART of the reason to be luck. Thus, with baseball players, the rule is the following: as long as a player has better than average statistics, you should expect them, in the future, to perform worse than they have to this point, but better than league average. How much weight you give to league average vs. player’s history should depend on the sample size of the player.
So, Pujol’s career stats get more weight than a rookie of the year’s.
-——————————————-

I hope that helps. As CMath will correctly point out, there are other factors in play, such as the fact that 28 year olds perform better, on average, than 20 year olds or 40 year olds.

by snolls on Jun 18, 2009 6:35 PM EDT up reply actions 0 recs

Mauer

Another problem with using this to say Mauer is probably likely to regress to below his career marks is a concept you explained earlier in your own essay, sample size. Mauer has quite a lot of data points at this point in his career, and his numbers have held reasonably consistent (oscillating around his career line) over that time. If it was his second or third year in the league we could see a greater chance for regression, but he’s got a pretty good track record built at this point, his numbers are getting to the point where we can trust them, outliers are getting beyond statistically likely.

Also, what cmath said. in addition to him regressing to the overall mean not being likely anymore, most players do follow a development curve. In this way, you could describe him as a coin that slowly becomes more unfair over time (and then goes back the other way eventually). It’s pretty well researched and measured ow this happens, and it follows with common sense (cents? coins? RIMSHOT PLEASE).

So while I think you do a good job of describing probability in general, I think your own mentions of sample size (and significant outliers becoming more and more unlikely with expanded trials) discredits your theory with regards to Mauer. I would not be surprised at all if Mauer is a true 335 or 340 hitter for a few years based on his track record and a standard development curve.

"You can't sit on a lead and run a few plays into the line and just kill the clock. You've got to throw the ball over the damn plate and give the other man his chance. That's why baseball is the greatest game of them all."
~ Earl Weaver
"In God we trust. All others must provide evidence."
~ Billy Beane

by AdamOnFirst on Jun 18, 2009 3:57 PM EDT reply actions 0 recs

Did someone say "rimshot, please"?

here ya go!

http://www.instantrimshot.com/

by by jiminy on Jun 18, 2009 5:18 PM EDT up reply actions 0 recs

Definitely

I agree Adam, though we can obviously debate the credibility associated with this sample size, and the amount of average improvement at this age.

Still, I expect him to be VERY good.

by snolls on Jun 18, 2009 6:37 PM EDT up reply actions 0 recs

My gut tells me .335

as Mauer’s “true” average at this point. The wild card is the newfound power. As discussed below, if he adds 20+ HR to his arsenal, with everything else staying the same, over 500 AB we’re talking about a 40 point increase in “true” average. What’s interesting is that (all else equal) his chances go up pretty dramatically if he maintains around a 30-35 HR pace.

by Adam Peterson on Jun 18, 2009 10:13 PM EDT up reply actions 0 recs

Also

Also, lets give Joe Mauer a whole bunch of these coins this offseason so he can do many many probability trials, or put them in his piggy bank, or purchase gum with them, okay Bill Smith?

by AdamOnFirst on Jun 18, 2009 3:59 PM EDT reply actions 0 recs

Amen. Many, many coins.

Doubloons even. Whatever he wants.

by Adam Peterson on Jun 18, 2009 10:14 PM EDT up reply actions 0 recs

I'm going with

a blank check

stop by Waving the Wheat (http://wavingthewheat.wordpress.com/) and The College Hockey Blog (http://thecollegehockeyblog.wordpress.com/)

by fetch9 on Jun 19, 2009 12:24 AM EDT up reply actions 0 recs

Here is a fact:

Ted Williams’ career average was .344, but he did have a .406 season. As a matter of fact, before he had that .406 age 22 season. He had a .327 and a .344 season. So he improved .062 from his previous best.

Mauer so far had a .328 and a .347 season. Not improbable….

We are not talking about Mauer being a .400 lifetime hitter, but it is probable that he would have a .400 season

by thrylos98 on Jun 18, 2009 8:17 PM EDT reply actions 0 recs

Make ma' haid hurt

Seems like a lot of coin flipping and numbers. Probability.

People don’t hit on paper.

Can Mauer hit .400? The catching thing is a problem.

But he’s the KIND of guy who has a shot. If Delmon, for example, were batting over .400, we would know he probably would not finish at .300, let alone at .400. You can’t swing at bad pitches and bat .400, despite the fact people like to say about certain players, “he’s a bad-ball hitter.”

The best hitter can’t consistently hit bad balls well in the big leagues.

Big-league batters who were considered “bad ball” hitters typically gained that reputation on inside pitches. You can beat up those inside pitches that are balls at times. Not so on the outside. You can’t give a pitcher two inches off the plate and expect to hit. 400.

When Mauer swings at a bad pitch – he did so Wednesday night – it’s usually an inside pitch.

Mauer has taken plenty of crap over the years for letting first-pitch strikes go by, for letting seemingly nice strikes go by later in the count. But the proof is in his performance.

Being selective is crucial to being a .400 hitter. It can not be done if you’re not selective. So, he’s the TYPE of batter who could hit .400.

Bloggin' the bloggers since 1938.

by Johnny Safron on Jun 18, 2009 11:38 PM EDT reply actions 0 recs

definitely the type

He is definitely the type who could, but I think there still has to be a lot of luck involved. The reason he is the type who could, is that his eye, decision making, and coordination give him a higher “true” average then others.

Now that this discussion is over, we can go back to rooting for him to do it, whether by luck or skill.

by snolls on Jun 19, 2009 1:49 PM EDT up reply actions 0 recs

What does luck comprise?

Bloggin' the bloggers since 1938.

by Johnny Safron on Jun 19, 2009 5:14 PM EDT up reply actions 0 recs

Luck

Good question… different interpretations of this are being used several places in this thread.

The stricter definition applied here has to do with batted balls missing fielders. Bloop hits dropping is good luck… hard hit balls right at a fielder is bad luck. That type of thing.

The much looser definition is the acknowledgement that baseball players are “streaky” and are able to play at a level above or below their ability for shorter lengths of time.

The Mauer situation is actually the “fun” version of this “problem”. Whether his true ability is .290, .310 or .340 or .380 or whatever, Joe’s going to be the starting catcher and he’s playing great.

The flip side of this same problem is the much more interesting one for me as a baseball fan. That is, how long do stick with the slumping regular before benching him. A lot of people are ready to stick a fork in Delmon lately, but there’ve also been cases where the young slumping player gets benched or sent to the minors too quickly leaving fans frustrated that the player is being “jerked around” and needs to be “freed”.

Telling the difference between “slump” and “suck” is a big deal in player management.

by DavidRF on Jun 20, 2009 7:03 PM EDT up reply actions 0 recs

If you take everything involved in baseball...

…and factor in luck – and there is both good luck and bad luck – luck would constitute a helluva small percentage of all the elements involved in an outcome.

The Wall Street Journal just published a tidbit on research done by two business professors “to account for the role luck plays in professional golf.”

Seems like a bunch of crap to me.

Phil Mickelson’s reaction on the course to his wife’s cancer diagnosis, for example, seems to play no role in their “statistical model.”

These two also claim, “If being on the leaderboard at the end of the tournament was due entirely to skill, we would see the same names every week.”

But we do see them every week. Tiger Woods is there because of luck?

And no one said it was “due entirely to skill.” There are physical challenges – bad shoulders, bad wrists, bad fingers. Mental burdens. Elements.

Geez, it’s not roulette, which involves no physical skills at all.

Bloggin' the bloggers since 1938.

by Johnny Safron on Jun 21, 2009 1:06 AM EDT up reply actions 0 recs

Still disagree

When I describe luck in baseball, I mean that these are random events, whose outcome will not necessarily equal the “ex-ante” (before the fact) probability.

Roulette is a physical skill, in the sense that how hard the operator spins the wheel is exactly what impacts the outcome. We just see it as a random process because, whether by lack of technical competence or simply chaos theory at its finest, we can’t predict the outcome.

The same is true for baseball. The best baseball player of all time, Babe Ruth, could not hit a home run every time he came up. Even in the HR derby players can’t hit it out every time. This is because there are certain elements that are out of their complete control, so random events happen.

That is what I mean. A player having statistics that are not reflective of their “true” ability is the luck I’m talking about. So, when you project a player’s future performance, you have to try to consider 3) elements:
1) what is their “true” ability? This should be based on a credibility weighted assessment of their past performance.
2) Will their “true” ability improve or decline? These are the factors that CMath focuses most heavily on. They are real. This would, in statistics, be called the “changing parameters”.
3) How will the random nature of their future events compare to their “true” ability. In theory, because it is random, it isn’t knowable. That said, one should consider that things won’t happen in real life, like they should on paper.

ON MICKELSON:
I don’t really care that Mickelson thinks those professors are full of crap. In fact, I’d like him to explain to me how his wife’s diagnosis impacted his performance. You could get into trying to measure the impact of sleepless nights, amount of focus, etc, but my guess is that at the end of the day, some players in the same position will perform well, and some poorly. And that the same player, put in the same situation multiple times, will sometimes perform well, and sometimes poorly. We all remember the times that a player gets sick and shows a “courageous” performance. On the other hand, we also all remember the times a player was sick, and just didn’t play well because of it (I know Kobe has had both, for instance).

by snolls on Jun 22, 2009 2:34 PM EDT up reply actions 0 recs

Very interesting post.

You have a very good way of relating to the material you were speaking of.

"He didn’t call me or anything. It was an accident, but a lot of people would have called to see how someone is doing after they got hit in the head. Especially if they had to go on the DL." — Morneau on pitcher Ron Villone after an April 2005 beaning.

by Gonzo2 on Jun 19, 2009 1:52 AM EDT reply actions 0 recs

stupid work

I do it too frequently at work. Usually at work my examples relate to roulette (people like gambling), but coins are more reflective of batting average (two outcomes, lots of trials).

by snolls on Jun 19, 2009 1:52 PM EDT up reply actions 0 recs

what do you do?

"I don't care about feelings." - Lou Piniella

by natetheskate on Jun 20, 2009 2:17 PM EDT up reply actions 0 recs

Actuary

I work in insurance. Another good analogy to this would be car accidents. We all know good and bad drivers, but as of yet, no-one has been able to demonstrate how you figure out exactly when and where a person will get in an accident (despite significant investment, I might add). The fact is, that it is partly random, but you will find certain factors that are highly correlated with outcomes. This is just like coins and roulette wheels.

by snolls on Jun 22, 2009 2:36 PM EDT up reply actions 0 recs