Comments on: Why Picking Your Projection System Matters

By: BobbyRoberto

BobbyRoberto — Tue, 08 Jan 2013 23:25:13 +0000

If you’re choosing one set of projections, the actual projection doesn’t matter as much as how the players rank by those projections. In the example of the Bill James’ Pujols projection, Pujols is projected for 1247.8 points. What’s important is how that point total compares to the replacement level first baseman in the Bill James projections.

If all of James’ projections are high, then the replacement level first baseman will be high also, so the difference between Pujols and the replacement level could be the same as any other projection system.

Whether you use points in an Ottoneau league or z-scores, the important thing is the difference between that player and replacement level.

By: bjoak

bjoak — Tue, 08 Jan 2013 21:24:08 +0000

To mcbrown,

The point is more about how some systems have strengths. For example, I know based on my results that my system is exceptional at projecting batting average and it is intuitive to me because I have a unique system for doing it.

Other systems view it more as a fluctuating little part of OPS rather than an all important fantasy stat and just use some weighted mean of a few years to get it.

Similarly, Shandler puts a lot more care into his SB projections than non-fantasy systems. Oliver is built around good MLEs and is best for young players. Etc.

When you average you lose the individual strengths of those systems.

By: Travis L

Travis L — Tue, 08 Jan 2013 20:07:56 +0000

Or by using BJ projection as the optimistic ceiling.

I generally look to ZiPS for low/middle end projections and BJ for the high end. Gives me a sort of spread around their projections. If I’m unsure I’ll consult Steamer or PECOTA.

By: mcbrown

mcbrown — Tue, 08 Jan 2013 19:54:12 +0000

Disagree completely. For fantasy owners, the main value of a projection system is taking emotion out of player evaluation. Averaging several systems makes it less likely you’re going to latch onto a particular outlier that conforms to your personal bias (“I’m feeling a breakout for Joe Schmoe this year and ZiPS agrees!”).

By: Bill James

Bill James — Tue, 08 Jan 2013 19:37:28 +0000

Agree with Chad on the point that each system will have strengths and weaknesses and it is important for owners to know those. But averaging each system is a really bad idea for reasons previous posters have pointed out. Here is a helpful link to learn the basics about projection systems:

http://www.fangraphs.com/library/index.php/the-projection-rundown-the-basics-on-marcels-zips-cairo-oliver-and-the-rest/

It is also important to consider what projection systems are using for their run environment. There is a great article here explaining the details of this thought process:

http://www.hardballtimes.com/main/fantasy/article/the-crazy-bill-james-projections/

The point is, each projection system needs to be taken into context. If we are expecting Albert Pujols to put up the exact points projected by James in this article, we may be disappointed. But if we value him with the James system and know he is worth incrementally more than the next best 1b, it doesn’t really matter what the ‘total points’ projected is. The point is to find value.

Finally, if readers would like to learn more about projection system accuracy, check out these links (although Tango link above is more recent I’m aware of):

Most recent study: http://www.insidethebook.com/ee/index.php/site/article/testing_the_2007_2010_forecasting_systems_official_results/
BP (2007): http://www.baseballprospectus.com/unfiltered/?p=564#34764
Chone (2006): http://lanaheimangelfan.blogspot.com/2006/12/pecota.html

One thing to note is when the Pecota (BP) system was at it’s best Nate Silver was at the helm and he is no longer associated with the system.

By: gweedoh565

gweedoh565 — Tue, 08 Jan 2013 19:26:57 +0000

One reconciles this by not using BJ’s projections.

By: Mike Podhorzer

Mike Podhorzer — Tue, 08 Jan 2013 18:55:23 +0000

Yeah. For fantasy owners, the value from a projection system is from the outliers. If you average them all together, suddenly you have the same projections as everyone else, safe and comfortable. That won’t help you win your league. It’s the projections that make you go the extra dollar or drop out of the bidding early that are valuable, and they disappear when averaging the systems.

By: Mike Podhorzer

Mike Podhorzer — Tue, 08 Jan 2013 18:53:12 +0000

My system :-) But seriously, I like Steamer the best for pitcher projections. Last year, I noticed I tended to agree with their set the most for individual pitchers. I also like their process since they take additional variables into account, such as fastball velocity.

By: Marver

Marver — Tue, 08 Jan 2013 18:37:13 +0000

I’ve done this in the past using some simple z-score analysis; basically, measuring likelihoods for each z-score bin for each projection system. This is one of the few applications where I get Bill James’s projections to not have a 0 coefficient.

By: bjoak

bjoak — Tue, 08 Jan 2013 17:53:53 +0000

It is surprising to me that so many people argue for combining systems. I know as a creator of a projection system, my outliers are different for a reason and at the end of the season they are just as likely to be right as wrong. But when you have a good outlier, you can get that player as a discount. By combining systems, you don’t have outliers you can get at a discount.

It would be an interesting study to see how correct the various systems are in their outliers.