Comments on: My Small Sample Size Plea http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/ Daily baseball statistical analysis and commentary Sun, 12 Feb 2012 23:34:21 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: John http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-71068 John Wed, 08 Apr 2009 23:43:20 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-71068 Jayhawk Bill - If Cash lived in a vacuum ur point wuld be a good but if you have a league with 270 starters and a bunch of bench players, 5 or 10 of them will be hitting 150+ pts over their head in April in BA, so Cash's start is just probably an anomoly caused by having hundreds of guys playing April. If it was the end of May, its certainly a good sign for Cash at that point cuz it goes from like 1% to like .1%. Jayhawk Bill -

If Cash lived in a vacuum ur point wuld be a good but if you have a league with 270 starters and a bunch of bench players, 5 or 10 of them will be hitting 150+ pts over their head in April in BA, so Cash’s start is just probably an anomoly caused by having hundreds of guys playing April. If it was the end of May, its certainly a good sign for Cash at that point cuz it goes from like 1% to like .1%.

]]> By: Rally http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70984 Rally Wed, 08 Apr 2009 13:21:52 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70984 10 team league, 23 players per team, so only the top 3rd of players are worthy of roster spots in the league. At the end of 2007, Lee was not one of them. 10 team league, 23 players per team, so only the top 3rd of players are worthy of roster spots in the league. At the end of 2007, Lee was not one of them.

]]> By: Kincaid http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70869 Kincaid Tue, 07 Apr 2009 21:36:42 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70869 The problem with that is that there are enough players and seasons in MLB that aberrations will happen. Things that have 1% probabilities will happen, as we'd expect them to with hundreds or thousands of chances. So when you wait for them to happen and then pick out the aberrations, you can't really assign much statistical meaning there because of the sample bias. If you randomly selected a sample, or if you pick a player beforehand and say you will look at his next 40 ABs, you could analyze the meaning with binomial theory, but by specifically selecting an anomaly out of the thousands of possible samples you could have picked after the fact, you are introducing a bias that makes the statistical implications meaningless. 99% doesn't mean all that much when your selection process heavily favours the 1%. The problem with that is that there are enough players and seasons in MLB that aberrations will happen. Things that have 1% probabilities will happen, as we’d expect them to with hundreds or thousands of chances. So when you wait for them to happen and then pick out the aberrations, you can’t really assign much statistical meaning there because of the sample bias. If you randomly selected a sample, or if you pick a player beforehand and say you will look at his next 40 ABs, you could analyze the meaning with binomial theory, but by specifically selecting an anomaly out of the thousands of possible samples you could have picked after the fact, you are introducing a bias that makes the statistical implications meaningless. 99% doesn’t mean all that much when your selection process heavily favours the 1%.

]]> By: Eric Seidman http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70842 Eric Seidman Tue, 07 Apr 2009 19:36:00 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70842 We're somewhat talking about different things here, Jayhawk, but let me be clear that you hit the nail on the head. It's just that the nail belongs to another topic. What you're discussing is how a small sample can change a player's true talent level. This is definitely true, and I wrote an article last year discussing Cliff Lee and CC Sabathia, using The Hardball Times' marcel in-season estimator to show the effects. Basically, you couldn't have had a better first month than Lee or a worst first month than Sabathia, but even with those opposite extremes, their in-season adjusted projections were not drastically different. What I'm talking about here is that people should not go nuts over someone getting off to a 6-10 start. Adam Lind could very well be hitting .400/.500/.900 with 12 rbis through 4 games... my point is - who cares!? It's 4 games. We’re somewhat talking about different things here, Jayhawk, but let me be clear that you hit the nail on the head. It’s just that the nail belongs to another topic. What you’re discussing is how a small sample can change a player’s true talent level. This is definitely true, and I wrote an article last year discussing Cliff Lee and CC Sabathia, using The Hardball Times’ marcel in-season estimator to show the effects. Basically, you couldn’t have had a better first month than Lee or a worst first month than Sabathia, but even with those opposite extremes, their in-season adjusted projections were not drastically different.

What I’m talking about here is that people should not go nuts over someone getting off to a 6-10 start. Adam Lind could very well be hitting .400/.500/.900 with 12 rbis through 4 games… my point is – who cares!? It’s 4 games.

]]> By: Chris http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70827 Chris Tue, 07 Apr 2009 18:32:29 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70827 I agree Jayhawk. I think the point is don't do anything too hasty based on one hot streak from an out of nowhere free agent. Your point about Cash is well taken, but by the same token anybody who went out and picked him up from the waiver wire didn't exactly reap dividends post-May 11th. Most of us are pretty savvy drafters and owners. Our presence on this very site indicates we do our research. If you've already researched your drafted players to death already, I think it's a mistake to scrap countless hours of analysis based on one sexy box score. I agree Jayhawk. I think the point is don’t do anything too hasty based on one hot streak from an out of nowhere free agent.

Your point about Cash is well taken, but by the same token anybody who went out and picked him up from the waiver wire didn’t exactly reap dividends post-May 11th.

Most of us are pretty savvy drafters and owners. Our presence on this very site indicates we do our research. If you’ve already researched your drafted players to death already, I think it’s a mistake to scrap countless hours of analysis based on one sexy box score.

]]> By: JayhawkBill http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70808 JayhawkBill Tue, 07 Apr 2009 17:48:49 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70808 Small sample sizes do have meaning, though. I like to apply binomial theorem to players' small sample sizes to see how significant they are. For instance, last May 11 Kevin Cash was batting .375. Obviously he wasn't a true .375 hitter, but a quick check of probabilities indicated that it was 99% likely that he was a true .207 hitter or better. Entering 2008, Kevin Cash looked more like a true .180 hitter, so that .375 batting average was still good news, even when discounted appropriately for sample size. Nay sayers will comment that he batted only .167 the rest of the way. I'd counter that catchers always tire later in the season, and that .167 was higher than his previous career split for June-September...the outstanding hitting in April and May was an accurate signal that he'd improved as a hitter. Other small sample sizes can have similar meaning, if they are dramatically far from the player's previous norm and if they're appropriately discounted for their size. Small sample sizes do have meaning, though.

I like to apply binomial theorem to players’ small sample sizes to see how significant they are. For instance, last May 11 Kevin Cash was batting .375. Obviously he wasn’t a true .375 hitter, but a quick check of probabilities indicated that it was 99% likely that he was a true .207 hitter or better. Entering 2008, Kevin Cash looked more like a true .180 hitter, so that .375 batting average was still good news, even when discounted appropriately for sample size.

Nay sayers will comment that he batted only .167 the rest of the way. I’d counter that catchers always tire later in the season, and that .167 was higher than his previous career split for June-September…the outstanding hitting in April and May was an accurate signal that he’d improved as a hitter.

Other small sample sizes can have similar meaning, if they are dramatically far from the player’s previous norm and if they’re appropriately discounted for their size.

]]> By: Chris http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70802 Chris Tue, 07 Apr 2009 17:44:43 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70802 Personally, I'd prefer a cool 324 out of Felipe Lopez. Personally, I’d prefer a cool 324 out of Felipe Lopez.

]]> By: Chris http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70800 Chris Tue, 07 Apr 2009 17:43:35 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70800 I have no problem with it, and I'm a Yankee fan. These guys are trying to squeeze a story out of each and every game of the year -- this is the angle they chose for an Opening Day Yankee loss. However banal and unimaginative it may be, it is valid. I have no problem with it, and I’m a Yankee fan. These guys are trying to squeeze a story out of each and every game of the year — this is the angle they chose for an Opening Day Yankee loss.

However banal and unimaginative it may be, it is valid.

]]> By: David Appelman http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70798 David Appelman Tue, 07 Apr 2009 17:17:42 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70798 Weird. That's exactly what they said on the Seattle postgame show! (they might have been serious) Weird. That’s exactly what they said on the Seattle postgame show! (they might have been serious)

]]> By: rwildernessr http://www.fangraphs.com/blogs/index.php/my-small-sample-size-plea/#comment-70796 rwildernessr Tue, 07 Apr 2009 17:16:29 +0000 http://www.fangraphs.com/blogs/?p=4063#comment-70796 Small sample size, schmample size. I think it is obvious to everyone that Ken Griffey Jr. is clearly going to hit 162 home runs this year. Small sample size, schmample size. I think it is obvious to everyone that Ken Griffey Jr. is clearly going to hit 162 home runs this year.

]]>