Some Thoughts on Monday’s Polling

Player projections, because of what they are, are always going to be pretty accurate at the group level, or else they’d be really lousy player projections in dire need of improvement. So when thinking about the future, a projection should always be the starting point. One truth is that there will be few exceptions to the rule that players follow their projections. Another truth is that there will always be exceptions. A third truth is that baseball fans will over-identify presumed exceptions, because we’re not very good at weighing recent events. As such, if this counts as a debate, it’s going to be a long-running, unwinnable debate, with new possible exceptions submitted every year. There’s always going to be some reason to believe in a given player who hasn’t performed like his projection over a couple weeks or months.

Dave wrote about some pretty important research last Friday, finding that in-season projections work well and there’s not much sense in isolating season-to-date statistics. Monday I put up ten polls and then another ten polls, focusing on position players and pitchers who haven’t met their projections so far. On each player, I wanted to gauge audience opinion, to see who people think might be exceptions, and who people think are just players on a streak or a slump. There’s not a ton we’ll be able to do with the results, when it comes to furthering our understanding of the game, but I thought it could be fun to quickly review some stuff we can already see. While more voting has taken place since the writing of this post, the numbers shouldn’t have changed very much.

Let’s start with a graph, showing poll results for each player. The graph is presented as an oversimplification, but the idea is this: voters either believe more heavily in the season performance, or they believe more heavily in the rest-of-season projection. There’s nothing in here about magnitudes, but this should provide an idea of which players the audience believes have meaningfully changed their true talent, with the projections not yet having caught up.

performancevsprojectiongraph

Let’s use Domonic Brown as an example, since he (unfortunately) shows up first. He entered Monday with a .257 wOBA, and he also had a rest-of-season projected .331 wOBA. Of the voters, 33% thought the projection seemed right. That left 67% who thought the projection looked too high — that is, they figure Brown is worse. There’s no indication of how much worse, since it was a subjective poll left up to judgment, but suffice to say, two-thirds of the voters believe Brown is bad enough now to not be that close to a .331 true talent. Last year, his wOBA was .351, and he’s still just 26 years old, but there seems to be a strong belief that his swing is messed up. He has been hitting way too many balls on the ground, for him.

The graph is arranged in descending order of audience weight on the in-season numbers. So at the far end, we get Nelson Cruz, in whose surprising performance the audience believes the least. Monday, Cruz was projected for a .355 wOBA. Last season, he posted a .359 wOBA. The voters don’t seem to be buying his power streak. If I had to guess at reasons, he’s almost 34 years old. His ball-in-play rates look normal, his discipline numbers look normal, and nobody’s written anything about any adjustments he’s made. And to be honest, I think there’s been Cruz-related skepticism around here for several months, and I imagine that’s lingering, hot start be damned. Usually hitters in their mid-30s don’t suddenly get a lot better, and that’s the audience opinion.

I’m more surprised by the name beside Cruz. I would’ve figured Lonnie Chisenhall would get a bit too much credit, being a younger player who was also in danger of becoming a busted prospect. With that kind of player, fans are often over-eager to believe in performance improvements. Fans figure he’s above-average, but they don’t figure he’s good, with 79% believing the .342 projection. With a BABIP in the .400s, it’s clear that Chisenhall’s numbers are going to come down. The projections, though, also see an increase in strikeouts and a decrease in power. I’m curious about the poll results here, but pleasantly surprised by the rational objectivity.

There are four players who the majority of fans believe have different true talents now than their projections:

  1. Domonic Brown (67% vs. 33%)
  2. Dallas Keuchel (66% vs. 34%)
  3. Brian Dozier (63% vs. 37%)
  4. Mike Moustakas (60% vs. 40%)

Just missing, at this writing: Felix Doubront, with splits of 48% vs. 52%. I’ve already touched on Brown. Keuchel isn’t surprising, here, because we’ve written about a repertoire change he’s made. My sense is people are more willing to believe in changes if they can easily identify a reason behind an improvement or decline. The same thought applies to Dozier, who made some mechanical adjustments to his swing a year ago. It’s easy to believe Dozier is better now, and that the projections make too much of his terrible and light-hitting 2012 debut. He’s projected for an ISO even below what he did in 2013. And then there’s Moustakas, who’s been frustrating people for years. Dave did write something about Moustakas making too much bad contact, and for that reason and others, people don’t believe he’s as good as even a .312 wOBA. I mean, two-fifths of voters believe he’s at that level, but 50% more voters believe he’s meaningfully worse.

To split these results ten by ten: with position players, on average, 41% of voters saw a change in true talent, while 59% believed the projections. With pitchers, on average, 42% of voters saw a change in true talent, while 58% believed the projections. I expected the pitchers to generate different results, because I think it’s easier to believe in a pitcher making a change than in a hitter making a change. Hitting, after all, is reactive, and it’s easy to see a repertoire change or a drop in velocity. But my expectations have been proven to be off, at least with these particular players.

To split these results ten by ten, differently: with the over-performers, on average, 41% of voters saw a change in true talent, while 59% believed the projections. With under-performers, on average, 42% of voters saw a change in true talent, while 58% believed the projections. Once again, no meaningful difference between groups, no evidence of potential biases. I could see it being more tempting to believe in short-term over-performance, but I can also see it being more tempting to believe in a player falling apart because frustrating is hard to shake and images of under-performance linger. Again, this could just be about the players I selected. Ultimately, we’re dealing with only 20 names.

Overall, the averages were 41% and 59%. Which means, overall, most people trust the projections. This, a few days after a post literally titled “You Should Trust the Projections”. On the one hand, that’s 41% of people disobeying the advice, but on the other hand, we know there are always exceptions, and these are 20 of the potentially most likely players to be exceptional. So even around the extremes, more people believe the projections than don’t. Seems about right to me. Seems like a sign of a pretty sharp audience.




Print This Post



Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.


30 Responses to “Some Thoughts on Monday’s Polling”

You can follow any responses to this entry through the RSS 2.0 feed.
  1. tz says:

    I think the biggest area where the projection systems struggle is for guys who hit very well in the minors but struggled their first few years in the majors. So when a guy like Dozier starts to really break out (like Lucroy is also doing), it’s tougher for a projection system to account for than it is for a knowledgeable follower of the game.

    Vote -1 Vote +1

    • Sky Kalkman says:

      Pretty sure most popular projection systems incorporate minor league performance.

      Vote -1 Vote +1

      • tz says:

        I know they do, but to clarify my initial comment, I think that projection systems would have a hard time determining how much weight to give to the recent MLB experience vs. the earlier minor league performance. Some guys make a smooth transition to the majors, others tend not to.

        Vote -1 Vote +1

  2. Andrew says:

    Honestly, after following Dave’s twitter battle with Keith Law, it left a bit of a bad taste in my mouth. I can’t believe that anyone would be so blind as to believe that any projection system is even remotely useful without real world, eye test scouting of the players. They are both tools for player evaluation and must be used in concert. It is absolutely silly to suggest otherwise.

    Vote -1 Vote +1

    • bluejays49 says:

      Yep.

      Projections are based off past performance. We have evidence that a player’s talent level has deviated from what his past performance would suggest.

      It should then be reasonable to suggest his projection is less than accurate.

      Vote -1 Vote +1

      • hebrew says:

        @Bluejays49

        your comment kind of ignores how important sample size is. Projections are based on, usually, massive sample sizes from years of performance, whereas what you refer to as a “player’s talent level” (another misnomer – I’d argue a players “talent level” never changes, but that’s for another time) is usually just from a small set of numbers, such as from this year only.

        it is highly unusual for a player to make drastic and wholesale improvements when his history suggests otherwise. If I ahve 5000 at bats across the minors and majors that show i am a .280 hitter with a .320 OBP and 14% LD%, and this year through 500 ABs i am hitting .220 with a .260 OBP and 11% LD%, have I truly forgotten how to hit, or am I just going through a bad stretch?

        You seem to be arguing that my “Talent level has deviated from my past performance,” when I really am far more likely to be just going through a bad stretch that will be offset by a period of hitting .320 with a .390 OBP and 20% LD rate.

        Vote -1 Vote +1

        • bluejays49 says:

          I agree with your point. Established players tend to perform as their careers suggests they should.

          What I am saying is that the projection of a player who scouts say has recently made fundamental changes to his game (for a pitcher, the repeatability of his mechanics, his velocity, the tightness of his breaking ball, etc.) will not be accurate. It does not incorporate all of the information relevant to projecting the player.

          Jimmy Nelson is the case in point here. Keith Law says that he is a fundamentally different pitcher than he has been in years past. He has altered his talent level. The projection system is using those past years to project his performance. If we are to take the scouts at their word (and why shouldn’t we), then Nelson’s projection shouldn’t be taken seriously.

          If Nelson were performing better than in years past at AAA yet talent evaluators were not seeing notable changes to his mechanics or pitches (as is the case with almost all pitchers), then I would have confidence in referencing Nelson’s projections.

          Vote -1 Vote +1

    • nd says:

      Is there a good place to see that twitter battle in its entirety?

      Vote -1 Vote +1

    • AK7007 says:

      To be fair Andrew, Keith Law was asking why you wouldn’t replace Estrada, and Dave gave a reasonable answer. Reasonable means that it’s a non-crazy answer, not that it’s the 100% always right answer. Estrada isn’t this bad, and Law should be able to see past small sample size noise. HR rate is unlikely to stay this way going forward.

      All Law would say is “your projections are dumb” without anything to back it up. Maybe scouts can be all magical and better than the projections, but Law didn’t say how. Looking at pitch f/x, Nelson’s/Estrada’s pitches (movement and velocity) haven’t changed all that much, so I don’t know what he has that significantly changes their projection. We know projections can be occasionally wrong, that’s what the recent string of posts has been about. We don’t know that we can reliably figure out which ones are wrong, but Law claims to without saying how. I’m not saying it’s not possible, but I’m saying show me the proof.

      Vote -1 Vote +1

  3. Michael says:

    I don’t know how many people remember Dave McCarty, who was rated by BA 22 and then 16 pre-1992 and 93. Big guy, line drive/gap power, spent 13 years going back and forth between the majors and minors, with a AAA slash line in 666 games of .312/.400/.533. Couldn’t stick in the majors, seven years, never more than 397 PA (his rookie year). Once in a while he’d show a little something and you would think he’d finally figured it out.

    Vote -1 Vote +1

    • srpst23 says:

      and?

      Vote -1 Vote +1

      • Michael says:

        And, conventional wisdom, plus the best evaluations of the professionals are sometimes wrong, and even more so when you are a fan and really want to believe. How many times do you read projected trade talks involving a proven quality regular for prospects and all the fans say is “no, you can’t give up…….”

        Vote -1 Vote +1

  4. Jason B says:

    “A third truth is that baseball fans will over-identify presumed exceptions, because we’re not very good at weighing recent events.”

    That pretty much says it all in a nutshell. Of course, the exceptions that are correctly identified are loudly announced, those that are misidentified quickly forgotten.

    Vote -1 Vote +1

  5. PackBob says:

    “Most” is kind of a slippery word, in that 99%51%. That’s a big range, and on the surface with no other information, most of 1,000 people could mean 501 or 999. Most = 501 yea and 499 nay; most = 999 yea and 1 nay. Both Oakland at 42-28 and Baltimore at 35-34 have won most of their games.

    But I like most of Jeff’s conclusions.

    Vote -1 Vote +1

    • Jason B says:

      I don’t interpret “most” that way; rightly or wrongly, I would never say Baltimore has won “most” of its games at 35-34. I don’t use it to mean (50% + 1) and never read it that way.

      Vote -1 Vote +1

    • Nathaniel Dawson says:

      Yeah, it’s tough to see an interpretation of “most” as being a slight majority. While it’s used in many different situations, and could have different meaning depending on the context, “most” generally signifies a strong majority. Something like perhaps two-thirds or more. “When he swings, he makes contact most of the time” wouldn’t suggest a player that is barely over 50% in contact rate. “They win most of their games” wouldn’t imply a team just over .500.

      Vote -1 Vote +1

  6. AF says:

    As I said on the last thread, I think that using *updated* projections answers a straw-man question — is in-season performance 100% predictive of ROS performance — and obscures the more interesting question: is early-season breakout a fluke or a sign of real improvement? The way to get at this question would be to compare *pre-season* projections to in-season performance.

    Vote -1 Vote +1

    • Hawk Harrelson says:

      The difference between pre-season projection and rest-of-season projection is nominal. Look at Brian Dozier for example, even with his new level of performance this year and last his rest-of-season projection is a .317 wOBA which is actually worse than the .319 wOBA he posted last year. I don’t have the pre-season projection handy, but what could it have been, .305-.310 maybe? In any case the projection sees him as a league-average-ish hitter, whether if you use this season’s performance to inform your projection or not.

      Vote -1 Vote +1

  7. Fatbot says:

    Polls are more a measure of media convincing than anything else. Dozier is a great example. Most readers are sold on a breakout because of writer comments like “made a mechanical adjustment a year ago”. Really, mechanics are the reason he’s magically doubled his walk rate? Or was it the Twins would tell him “just swing less at strikes in the zone” as the easy path to success? Facts of Doz: in plate discipline the biggest changes are 3% less Z-swing% and an increase in O-contact% (which Moose teaches us = bad?). Meanwhile, only change in batted ball distribution is 2% less line drives replaced by 2% more ground balls, which somehow translates to a doubling of HR/FB%? How about his avg. FB distance justifying it? Went from 184th (276ft) last year to 126th (279ft) now? So Doz summary: more contact on bad pitches while swinging less at strikes with no improved zone contact = profit. Same FB% but 2% less LD and 2% more GB, his FB distance “surged” a grand total of 3 feet = profit. He does have the pedigree of an 8th round pick just deciding to break out with more walks and ISO than ever in his life, including minors. So kneel before Doz (which of course is Zod spelled backwards)!

    Vote -1 Vote +1

    • Nick says:

      I think his approach is to wait for a pitch he can pull out of the park; all of his HRs are to left field and when he hits it to right or center, it doesn’t usually go very far which could explain the low overall distance relative to the HR/FB%.

      The increased BB rate is likely because he’s extremely patient, which is prob a good thing considering he doesn’t have the best eye (as evidenced by the poor O-swing and Z-swing % you eluded to). But he doesn’t swing and miss; I’d bet he has more backwards Ks than forward ones given his very low SwStr%. Pitchers may also be pitching him a little more carefully this year given his increased power: according to Pitch F/X, his zone% is down 5% from 2012-13.

      Vote -1 Vote +1

    • Tim says:

      Or was it the Twins would tell him “just swing less at strikes in the zone” as the easy path to success?

      I’m quite confident the Twins would never say that to anyone.

      Vote -1 Vote +1

  8. Kris says:

    If Dave would have titled Friday’s article “You Should trust the projections, over a 162 game sample of 50-60 players” it would’ve saved everyone a lot of trouble.

    Vote -1 Vote +1

  9. Dr. Mantis Tobaggon says:

    I don’t think people have a problem with the concept that most players will perform closer to their projected (read: regressed towards mean) level the rest of the season. If you take a large enough sample of players, the total results almost certainly will bear that out. I think what people are wondering if what do such projections tell us? Anyone can regress a group of players back to the mean, and they will probably be right on aggregate, but what use is such a system.

    To put it in an analogy: I predict the 2016 Presidential election will end with a Democrat winning 300-238, predicting the various state outcomes as well. The election rolls around, and the Democrat wins 290-248, but only 20 of my 50 state predictions turn out correct. It seems that in the way Fangraphs is measuring the accuracy of baseball projections, my election projections would be looked upon favorably, because I was only slightly off in my total amount, yet in reality, they would be horrible projections, as I missed a ton of states and got bailed out by the fact that most elections end reasonably close.

    Vote -1 Vote +1

    • AK7007 says:

      That would imply that most of the players end up being much different from their projections, but the aggregate numbers are alright. In reality, most players are close to their projections, with outliers above and below. That’s what these posts are about, that most players end up near their projected numbers, and that we don’t really have an answer for those guys that missed. Yeah, on an individual level, after the fact you can say “mechanical change.” But what about all the players that made mechanical changes and then ended up staying the same? or got worse? those “player interest stories” were swept under the rug and forgotten because they don’t fit the sportswriter’s preferred narrative. While the ones that worked out are held up as an example of “we scouted this guy and saw he made changes for the better!” Every player is constantly making adjustments to get better, and we don’t know ahead of time which adjustments are going to lead to major differences.

      Vote -1 Vote +1

    • Nathaniel Dawson says:

      They don’t use the aggregate to test projection systems. They use the absolute error for each player. So if the projection has two players that are both projected to hit .340 wOBA, and one hits .300 while the other hits .380, the error for that projection system is .40. They don’t test the projection by averaging both players and say the projection was right on.

      Vote -1 Vote +1

  10. Cliff says:

    I wanna see a poll for Betances, goddamn it! MGL says no one can predict which players will be true outliers? Come on…

    Also would love to see the study done more granularly. I.e. how do the projections systems do when working off minor league numbers in general vs. major league numbers etc.

    Vote -1 Vote +1