A Dose of Reality for Prospect Watchers
All of the minor league regular seasons are officially over, and it will soon be one of my favorite seasons of the year — prospect evaluation and ranking time. I personally love reading the various scouting reports and rankings; I devour it as much of it as I can stand. Now excuse me while I become a wet blanket.
I think as we look at minor league stats and read scouting reports, if we are not careful we can get rose-colored glasses when it comes to our outlook of the future of these players. The truth is most of these players we find ourselves pulling for simply won’t ever make it.
Victor Wang has done some tremendous research about prospects and their value. In determining their value he had to find the rate of which players bust in each category he divided them into. This is a healthy dose of realism to keep in mind when we’re looking at the any one club’s farm system.
* 10% of top 10 hitting prospects bust.
* 31% of top 10 pitching prospects bust.
* 21% of top 11-25 hitting prospects bust.
* 32% of top 11-25 pitching prospects bust.
* 35% of top 26-50 hitting prospects bust.
* 33% of top 26-50 pitching prospects bust.
* 45% of top 51-75 hitting prospects bust.
* 39% of top 51-75 pitching prospects bust.
* 43% of top 76-100 hitting prospects bust.
* 43% of top 76-100 pitching prospects bust.
* 59% of ‘B grade’ hitting prospects bust.
* 52% of ‘B grade’ pitching prospects bust.
* 83% of ‘C grade’ hitting prospects bust.
* Around 75% of all ‘C grade’ pitching prospects bust.
Top 100 prospects are Baseball America’s. B and C grades are as ranked by prospect wonk John Sickels.
Top ten hitting prospects are just about can’t miss. Not all of them reach the level of stardom, but they seldom fail to bring value to a team. Going on down the line, the rates of attrition obviously get higher and higher. I find the failure rates among top pitchers to be striking, and it’s interesting to see how things even out between hitters and pitchers as you go down the grades.
This isn’t anything really new, but it is something to keep in mind when reading these rankings. Reading some reports you would be almost led to believe that even the majority of C grade hitting or pitching prospects will end up being at least a major league utility players or a middle relievers, but that’s simply not the case.
Feel free to soak up all the prospect hype you can find, but always take it with a grain of salt.
Print This Post
What’s the level of “bust” in these cases? Is it “fail to meet expectations of grades” or “fail to meet replacement level standard,” or is it something completely different?
See below, or follow the link — Wang explains it all in his piece.
I would like for someone, sometime, to explain the qualifications of John Sickels. With all due respect, when did Sickels’s grades become highwater for the industry? Is Sickels some guy who decided to run a website one day listing his opinions on minor leaguers, and for lack of alternatives, his website began growing? Can someone please explain? It seems like anyone could fool the public, no?
I respect Sickels’ opinions a great deal — but at the same time it’s just one man’s opinion no matter how qualified he is and further study using wider criteria would be a welcome addition to this discussion. Same thing goes for BA’s rankings. More than one opinion goes into that, but that’s still fairly narrow.
I prefer Sickels’ analysis over Baseball America’s any day.
Sickels is probably the best prospect evaluator in the industry. He used to be the main minor league columnist at ESPN for 9 years before starting his own gig. So yeah, I’d argue that he’s way more qualified than most people covering prospects today.
not even close.
he looks at numbers and that’s about it.
“oh, this guy has a lot of strikeouts”
‘must improve plate discipline’
sickels is phenomenally overrated as prospect guru.
I agree with Big Baby.
Sickels does nice work, and if he is on site, has some decent evaluation of tools, but the majority of his analysis comes from looking at numbers, which, when looked at in the context of prospect evaluation, mean almosst nothing.
BA is still the end all be all of prospect evaluation. They talk with scouts, who look at and watch players every day.
“majority of his analysis comes from looking at numbers, which, when looked at in the context of prospect evaluation, mean almosst nothing.”
I disagree with this 100% Eyes can be more deceiving than not. I prefer factual information (stats) over some archaic, subjective info (traditional scouting).
I’m not touting Sickels as the supreme prospect guru or anything but as an individual, he is more informed than anyone else I can think of. Who else other than Baseball America dedicates his entire body of work on baseball prospect?
Yoda,
But what do you say about a player who hits, say .290/.360/.400 in a hitters league, or racks up a lot of that total against AAA pitchers on their way down vs on the way up?
Or what about the pitcher who has a huge k/ip rate in AA ball but barely touches 90 mph because of his big breaking ball?
Sickels, although he will know about these quirks, will say that both players have nice performance and show promise. He LOVEs walk rates and equates it with patience in translating a player as prospect vs a guy with a big time future or bust. When a guy in any level of minor league ball can walk a lot because he faces a ton of bad controlling pitchers and doesnt know the first thing about going after “his” pitch. Traditional scouting makes up for all of that.
True, scouting is a subjective business and there are many failures, but those guys rarely miss and there is a reason for that. I like what Sickels does, but I look at BA or Kevin Goldstein first before reading what he has to say.
Traditional scouts rarely miss? They miss more times than not. The entire game of baseball is moving towards advanced statistics over traditional scouting. For good reason.
Sickels accounts for level, leagues, age, park, etc. so I’m not really sure where you are getting your information from. If you compare everyone’s track record, I’m willing to bet that Sickels, Goldstein, BA , etc all come out pretty evenly.
Sickels may come close to what BA or Goldstein say, but he could be using BA’s list and then just throwing in numbers. Like I say, I like his stuff, I just dont think he is as good as some people thing.
Scouting with the eye will never go away in this game. The A’s tried to put statistical evaluation over scouts eyes in building their farm system, and they failed. Not until they started to realize, and modify thier MO, did it turn around.
There is not anyone out there that the scouts have “missed” that didnt get drafted or make a MLB roster for a reason. There are guys who sign as Free Agents, who dont get drafted, but its very rare, and in the end, you still needed to impress a scout with your tools to get that deal. No one in the game has ever been “screwed” out of a spot on an MLB roster, this is because of scouts, they may miss on the low end, but not on the high, and that is what I mean by that statement.
if you think the opinions of scouts matters less than stats for minor leaguers, then you’re completely misconstruing the thrust of the statistical revolution.
I’m guessing you guys aren’t a big fan of Bill James either.
yep, i hate bill james. that’s why i come to fangraphs.
Honestly, it wouldn’t shock me if the pitcher attrition rate is the reaction to failure. You have to think most top prospects have never seen the competition of MLB hitters before, and also have never gone through long stretches of getting shelled.
This is all theory and hypothesis, though, could be worth checking into.
Plus the Top 10 is all relative. Each year is different, with differing overall strengths of hitters and pitchers. This is only one of 100s of examples but Brandon League was the Jays #1 prospect according to BA (2006 I believe) so yeah, take it with a grain of salt.
Hell, BA thought Joba was the be all and end all for pitching prospects. A guy who was widely considered one of the top pitching prospects, ever. He has certainly been above league average, but he still has a ways to go to live up to that kind of hype and praise.
I’ll wait until he ceases to be the most babied pitcher in the history of MLB.
And as a Sox fan, I’d love to see Youk hit a liner off him btw. Not, like, Bryce Florie style career ruining or severely injuring or anything, just the kind that may teach him to stop throwing at heads.
Sorry, random Joba rant, obviously I wouldn’t want to get a beer with the guy.
Funny, I’d love to see Joba drill Youk in the neck with a 97 mph fastball. Of course–like you said–not career threatening or anything, but just to give Youk something worth whining about.
The Red Sox do seem to have a sense of entitlement to close calls in the strike zone…
When Roy Halladay was pitching to V-Tek (could that C be any bigger btw?), V-Tek took time and walked around once, then after a close strike call made this big gesture and whined and stepped out again. Halladay sort of yelled at him (I read his lips, it was “what a bitch”).
After he struck him out (obviously), he made a motion with his hand, like swatting a fly away!! Roy never EVER shows much emotion (unlike the slack jawed Joba) on the mound but even the Sox found a way to irk him…
I have a different take on this information. When I see that the top ten hitting prospects have only a 10% bust rate compared to twice that for the 11-25 ranked prospects, I can’t help but be impressed with the ability of BA to assess young talent. If BA were not good at what they do then the bust rate curve would be a lot flatter.
It’s kind of interesting though, the pitcher curve is much flatter.
* 31% of top 10 pitching prospects bust.
* 32% of top 11-25 pitching prospects bust.
* 33% of top 26-50 pitching prospects bust.
* 39% of top 51-75 pitching prospects bust.
* 43% of top 76-100 pitching prospects bust.
* 52% of ‘B grade’ pitching prospects bust.
* Around 75% of all ‘C grade’ pitching prospects bust.
Kind of interesting that for even the top 100, it’s only a 12% difference, and 1-50, it’s essentially the same, whereas in hitting prospects, the top is much safer.
That’s how pitchers are. There’s a higher risk of severe injury, which probably explains a good deal of it, and it just seems that we have less capability of translating HS/college/minor league pitching performances to the major league level. The last part I can’t really explain, other than that there’s a lot more that goes into pitching than there is going into hitting + playing the field, not to mention the fact that you can mostly hide poor fielders, where the only place to hide poor starters is in the bullpen, which gets judged far more harshly than fielding.
Iwould agree with Michael. This would suggest to me an analytics gap between evaluation of hitters and pitchers.
Thank you, Erik.
What’s the bust rate for Sickels grade “A” prospects?
Heh, Jed Lowrie is certainly one. Lars Anderson is on his way to being another.
Calling a 25 year old SS with 369 PA’s “certainly” a bust? That might count as somewhat of a harsh assessment.
I second Mike’s question: What is the definition of a “bust”? This site is so mathematically oriented that this seems a reasonable request.
The methodology is all in Wang’s article.
A player was a bust if he averaged 0 or less Wins Above Bench (WAB, equal to WSAB/3) per year, a player was a contributor if he averaged between 0 and 2 WAB/year, a player was an everyday player if he averaged between 2 and 4 WAB/year, and a player was considered a star if he averaged over 4 WAB/year.
Right, folks just need to follow the link. The one question I have is how closely Wang’s WAB tracks with WAR — is 0 WAB truly replacement level, 0.0 WAR, or something higher?
I haven’t read all of Wang’s article, but I would guess that WAB has a higher baseline than WAR, just based on the logic of bench players being better than replacement level as a whole.
It looks to me like top 50 pitchers are almost as likely to succeed as top 10 but with hitters it is more predictable.
So it would seem to me I would be more inclined to take a hitter if drafting early and only take a chance on a pitcher if drafting later when the failure odds are the same either way. That’s really very useful info for a GM who doesn’t want to be seen as wasitn high draft picks.
The press and even teams tend to fall in love with guys who are going to be drafted high. This applies to free agents and other sports as well. It gets especially dangerous if the buying team has a need at that spot. Just because somebody is the best guy available doesn’t mean he is any good.
Doesn’t mean he isn’t either. But these stats really argue for the “grabbing every useful arm you can find” strategy of building a pitching staff. You get a bunch of quality arms and hope 1/4 or better become useful to better players. Drafting pitchers is dumb relative to a similarly ranked hitter. Pitching is a quantity draft hitting is a quality draft.
I wonder how Mr Wang accounts for guys rising up through the Prospect rankings. Brett Lawrie for instance started the year in the 75-100 range, and probably makes it into the top 50 next year.
I emailed him this very question… along with a link to this thread so maybe we get a direct response or i’d be happy to share any response i get to that effect.
I would imagine the proper way to evaluate the systems would be to include them in both ranges.
Well apparently Steven Strassburg has a 31% chance of busting. What are his chances of winning 20 games once? Probably very very small. That signing bonus sure looks crazy to me, and I root for the NATS!
Well others have shown that the top 1-5 draft picks are significantly better than even 6-10 so these numbers don’t really apply to Strasburg. Particularly since he was the unanimous number one pick and is considered better than the average number one pick at that.
Here’s a good graph showing this: http://baseballanalysts.com/archives/2009/06/draft_picks_and.php
I don’t know about that. The relatively consistent 30% bust rate for top-50 pitchers suggests to me that the bust rate is essentially the rate of career-altering/ending injuries, and I don’t think there’s any reason to think that a #1 pitching prospect is any more injury proof than #10 or #100. He’s probably much more likely to become a star if he manages to dodge the injury bullet, though.
I think one of the factors also rests in how close a guy is to the majors.
For instance, say Bumgarner and Strasburg are the top 2 pitching specs in baseball. I’d say that Strasburg’s chances of busting are lower than Bumgarner simply because he has a shorter path to travel.
And then there’s also the fact that Strasburg is a super beast.
As if I didn’t already feel good about Jason Heyward’s chances in the majors. And to DC Strasburg has a much much higher chance of busting compared to Bumgarner
I have no idea what “Wins Above Bench” are, but the very word “bench” and the way he judges “busts” implies that a 20 year career as a bench player is a player “busting.” At least that’s how it seems. Wang might be underestimating the value of a player who is simply good enough to be in the majors for an extended period. I have a hard time calling that guy a bust if he’s the 98th ranked prospect.