Where ZIPS and Steamer Disagree

With the ZIPS projections now loaded onto the site and the player pages, I thought it’d be fun to take a look at a few examples where ZIPS and Steamer — probably my two preferred projection systems at the moment — differ this year, and whether there’s anything in particular we can learn from those differences.

First, I wanted to whittle down the population of players that I was dealing with. While I appreciate Steamer’s pluckiness, 4,136 projections for position players might be a little bit of overkill. I, for one, am not overly concerned with how Jeyckol De Leon is going to perform this year. Maybe it’s just me.

ZIPS projects a slightly more sane number of position players — 1,046, to be exact — but even that is still a little unwieldy, as a good chunk of those guys are sub-replacement level minor leaguers who aren’t going to see the Majors this year. By and large, we care mostly about the projections for players who are going to see substantial regular playing time in the big leagues this year, or at least, I do. Carson can care about all the fringe prospects he wants; I’ll leave that to him.

So, in order to get a list of projections for guys we care about, I excluded players who had never been in the majors or were projected to be below replacement level, leaving us with 601 Major League position players. That was still a little unwieldy, though, so I took the top 180 players by the average WAR of the two systems, which gave us a good selection of players that are projected to be league average or better by one of the two systems.

From there, it was a pretty simple sorting task to find some big differences, but many of them are driven by assumptions about playing time rather than big gaps in the actual projections. For instance, ZIPS lists Brett Jackson as a +2.5 WAR player, while Steamer comes in at just +0.9, which seems like a huge difference of opinion until you realize that the ZIPS number is based on 626 PA and the Steamer number is based on 274 PA. Rescale them both to 600 PA, and the gap is just 2.4/2.0, showing that the two systems are essentially in agreement on Jackson’s overall profile for 2013.

Since I care more about the differences in the projected performances than in the playing time gaps, we’ll focus on the big discrepancies in projections per 600 plate appearances (or 450 PA for catchers). Putting things on the same playing time scale will inflate the numbers of part-time players and injured guys while depressing the numbers of durable iron men, but we’ve already gotten rid of most of the part-time guys with our filtering, so that’s a trade-off I’m willing to make. Oh, and thankfully, the two systems were already on almost identical scales for these players, so there was no need to make any adjustments to the numbers as we did with the Fans projections last week.

Long introduction finished… on to the differences. We’ll start with the 14 players where Steamer projects at least +1 additional WAR per full season (600 PA non-catchers, 450 PA catchers):

Name ZIPS WAR ZIPS WAR/600 Steamer WAR Steamer WAR/600 Difference
Justin Upton 3.3 3.0 5.1 4.7 1.7
Pablo Sandoval 3.4 3.7 4.5 5.0 1.3
Daniel Murphy 1.0 1.3 2.6 2.5 1.2
Wilin Rosario 2.6 2.5 3.5 3.7 1.1
David DeJesus 1.4 1.7 2.6 2.7 1.1
Michael Young 1.3 1.3 2.2 2.3 1.1
Paul Goldschmidt 2.5 2.3 3.3 3.3 1.0
Nick Swisher 2.4 2.4 3.5 3.4 1.0
Alex Avila 3.1 3.0 3.5 3.9 1.0
Allen Craig 1.9 2.3 3.0 3.3 1.0
Lorenzo Cain 1.5 1.9 2.3 2.9 1.0
David Wright 3.8 3.7 5.1 4.7 1.0
Jesus Montero 1.9 1.4 2.6 2.4 1.0

While it was pretty easy to spot patterns with the differences between the fans and Steamer, this is a pretty interesting variety of player types. There’s no regular in baseball that Steamer likes more relative to ZIPS than Justin Upton, as Steamer is projecting a .374 wOBA while ZIPS comes in at just .342. Steamer projects six more doubles, six more home runs, and 22 fewer strikeouts in nearly equal numbers of playing time, so this simply comes down to a difference in opinion on how well Upton’s performance will translate outside of Arizona.

The three guys behind Upton are all in their mid-20s and theoretically headed towards their primes, so if you stopped after the top four, you might assume that Steamer is more optimistic about young players improving than ZIPS is. However, you’ll see shortly that we can reject that hypothesis pretty easily, and after all, these four are followed by the likes of David DeJesus (33), Michael Young (36), and Nick Swisher (32), so there doesn’t appear to be an obvious biased towards a particular age range here.

If you squint, you can kind of see a pattern towards favoring guys who hit for power, but you have to ignore Cain, Murphy, DeJesus, and Young in order to see that pattern really stick. In reality, it seems like this is just a pretty big potpourri of player types, and Steamer has differing reasons for preferring them. With all the variables that go into projections — aging curves, park effects, differing amounts of regression on certain numbers — it’s natural that there are going to be places where two well constructed models diverge. The fact that we can’t easily identify a common reason for the difference might actually be considered a good thing.

Let’s move on and look at the players that ZIPS likes more than Steamer. Again, we’re looking at players where’s at least a +1 WAR difference per 600 PA (or 450 PA for catchers).

Name ZIPS WAR ZIPS WAR/600 Steamer WAR Steamer WAR/600 Difference
Jurickson Profar 3.7 3.6 0.7 1.2 2.4
Bryce Harper 4.7 4.4 3.0 2.7 1.7
Craig Gentry 2.5 4.5 1.1 2.8 1.7
David Ortiz 3.1 4.4 2.4 2.9 1.6
Giancarlo Stanton 6.4 6.7 5.6 5.5 1.3
Mike Olt 2.9 3.6 0.7 2.5 1.1
Brett Lawrie 4.5 4.7 3.4 3.7 1.0
Torii Hunter 2.8 3.0 1.8 2.0 1.0

And here’s why we can discard the notion that Steamer likes young players more than ZIPS. Of the eight players where ZIPS is more bullish, two of them are 20-year-olds, two others are 23-year-olds, and there’s a 24-year-old. And then there’s two 37-year-olds, just for fun. So, ZIPS likes both the very young and the very old more than Steamer. Or, at least, a couple of very young and a couple of very old players. With just 22 players where there’s a gap of +1 WAR per full season, we can’t really draw any hugely firm conclusions about the differences.

I do think it’s interesting that ZIPS goes along with the fans optimism about Harper and Profar, however. We noted that the fans were far more optimistic than Steamer in regards to players with very short Major League track records, but ZIPS is actually quite a bit more bullish on these two as well. To see if this was a persistent pattern, I went and looked at the projections for players who had never played an MLB game, and ZIPS was consistently higher for most of them as well. There are 15 minor leaguers that ZIPS projects out to +2 WAR/600 or better for 2013, and ZIPS is higher on every single one of those 15 than Steamer is.

So, between ZIPS, Steamer, and the Fans, it seems like Steamer is at the low end on prospects and very young players, ZIPS rests in the middle, and the Fans are at the high end.

Of course, none of that explains why ZIPS is also higher on David Ortiz and Torii Hunter than Steamer, but it could be that Dan Szymborski is using a flatter aging curve than the guys calculating Steamer are. That’s just a wild guess based on two data points, though, so don’t take that guess seriously. As with before, it’s informational to know where the two systems differ, but don’t draw any conclusions about whether one is more right than the other based on these examples.

Many tests have shown that combining multiple projections is better than simply relying on one system anyway, so that methodology will smooth many of these gaps. If one good projection system thinks Upton is a +3 win player and another thinks he’s a +5 win player, then I’d be just fine assuming that he’s actually a +4 win player. The truth often lies in the middle ground. I suspect we might find that these players end up not too far from the number between their ZIPS and Steamer projections for 2013.

Print This Post

Dave is a co-founder of USSMariner.com and contributes to the Wall Street Journal.

24 Responses to “Where ZIPS and Steamer Disagree”

You can follow any responses to this entry through the RSS 2.0 feed.
  1. Kevin says:

    Great stuff. Will there be a similar article on pitchers?

    Vote -1 Vote +1

  2. Tchaikovsky08 says:

    Good article.

    What shocks me about ZIPS’ projections for J. Upton is the .263 avg. Is ZIPS putting *that* much stock into his Arizona home/road splits? I guess so. Seems awfully low to me given his career to date, especially for a guy who’s still just 25.

    Vote -1 Vote +1

  3. Jeyckol De Leon says:


    +13 Vote -1 Vote +1

  4. Le Vagabond says:

    Very Intresting article I’m intrested to see the article on pitchers and see if there will be a pattern for players being viewed diffrently in that reguard. Something Guthrie will be on that list, just a hunch.

    Vote -1 Vote +1

  5. byron says:

    I’d like to see everybody compared with Marcel, to start. Probably easier to see where systems are deviating from the middle more than against each other.

    Vote -1 Vote +1

  6. Dave S says:

    Would be very interesting to check outcomes for similar dis-similar players based on last year’s projections… no?

    Then we’d have some outcomes to examine.

    Vote -1 Vote +1

  7. AC of DC says:

    I assume the “flatter aging curve” for the likes of Ortiz and Hunter is laden with compliments and encouragement, particularly about how fit and youthful they appear to be, perhaps even featuring a sweet smile or playful wink.

    Thank you, thank you. You’ve been great. They’ll validate parking in the lobby.

    Vote -1 Vote +1

  8. jesse says:

    While I don’t see it here, when you compared the steamer/fans projections the biggest thing I saw repeatedly was that fan always had better babips for players. Maybe its still tje thought of the .300 being the average (at least for pitchers, with no base set for all players people need a base reference point) while in actuality babip are much lower than that. Is be curious if the fan projections simply has .010 taken off of babip how much that would bring them in line

    Vote -1 Vote +1

  9. Jaack says:

    I bet if ZiPS had run something on Jeyckol De Leon, it wold be more like Mr. Hyde. /rimshot

    Vote -1 Vote +1

  10. KilroySleptHere says:

    “Many tests have shown that combining multiple projections is better than simply relying on one system anyway”

    Now that FanGraphs has five different projection systems on player pages, have you considered adding such an average of all five? (Or four, if you wanted to exclude Bill James so you could keep WAR.)

    Vote -1 Vote +1

    • cody k says:

      or exclude Bill James because it doesn’t appear to use much regression and over-projects counting stats and general expected abilities

      Vote -1 Vote +1

  11. sambf says:

    “To see if this was a persistent pattern, I went and looked at the projections for players who had never played an MLB game, and ZIPS was consistently higher for most of them as well. There are 15 minor leaguers that ZIPS projects out to +2 WAR/600 or better for 2013, and ZIPS is higher on every single one of those 15 than Steamer is. – See more at: http://www.fangraphs.com/blogs/index.php/where-zips-and-steamer-disagree/#sthash.koHaAjsp.dpuf

    Good article, but this particular part has issues–when you select for young players ZiPS projects to +2WAR/600 you’re selecting for minor leaguers ZiPS happens to like–not a random sample of minor leaguers…

    Vote -1 Vote +1

  12. JC says:

    I was scanning through these lists to see what was accounting for the big differences. Almost all of the huge projection differences can be explained by gaps in projected BABIP (Profar, Harper, Stanton), defense (Gentry, Lawrie, Murphy), home run rate (Ortiz, Upton), or some combination thereof (Olt, Hunter, Rosario, Goldschmidt). But then I ran into this.

    David DeJesus
    Steamer: .330 wOBA, -3.0 Fld+Bsr (575 PA)
    ZiPS: .326 WOBA, -2.2 Fld+Bsr (507 PA)

    How does Steamer gives DeJesus an extra win over 600 PA? Does ZiPS have DeJesus playing in right field while Steamer has him in center?

    Vote -1 Vote +1

    • J. Cross says:

      I think that’s probably it. The Fans give DeJesus (on average) 12 games in left, 78 games in center, 42 games in right and 6 as utility outfielder. Since, CF is the most common, his replacement level (for Steamer) is based on CF.

      Vote -1 Vote +1

  13. equist says:

    Zips curently projects Adam Eaton to be HBP 21 times thins year which, if that did happen, would tie the current rookie record of 21 HBP by David Eckstein of the 2001 then Anaheim Angels (no LA necessary). This feat was accomplished in 664 PA.

    Thought you should know.

    Vote -1 Vote +1

  14. Rudy Gamble says:

    “Many tests have shown that combining multiple projections is better than simply relying on one system anyway, so that methodology will smooth many of these gaps.”

    Which tests are these and what are they testing? I’ve observed that averages are ‘safer’ and produce lower RMSE but just ends up serving the same function as regression. Might as well just use Marcel rates then…

    Vote -1 Vote +1

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current ye@r *