FanGraphs Baseball


RSS feed for comments on this post.

  1. This wasn’t clear to me from reading this article, but are CLIFFORD’s candidates and Marcel’s candidates the same group of players, or at least largely so?

    If they are, then yeah … stymied. But if it’s a wholly different set of players, then not so much

    Comment by David — January 23, 2013 @ 10:06 am

  2. No, I didn’t mention that but it’s a good point. Of the 34 CLIFFORD candidates, only 3 were also identified by Marcel. So, yes, there may be something there to investigate. I just haven’t gotten back to it. So, not completely a loser, but still thought the overall “failure” story might be worth telling.

    Comment by Bill Petti — January 23, 2013 @ 10:13 am

  3. Positive outcome publication bias does occur, but it leads to more interesting papers. To quote OCP executive Dick Jones, “Who cares if it works or not?”

    You could have also solved the lack-of-results problem rather easily in a well-established social science method delineated in this paper: .

    Result fabrication leads to much more interesting conclusions than “I have found nothing interesting.” (You can also run correlations of enough data to get a nice positive result; did you know that sales of organic foods have a brilliantly high correlation to autism diagnoses?)

    Applying effort to find the right answer is hardly going to get you a grant. As it stands, Lennay Kekua does not like your article.

    Comment by John R. Mayne — January 23, 2013 @ 10:20 am

  4. Wow, only 3? That’s very interesting, and definitely makes me think it wasn’t a failure at all. If you can identify an entirely different set of players poised to decline, then that’s fantastic!

    Comment by Matt Hunter — January 23, 2013 @ 10:26 am

  5. Based on what you’ve said, if you create a new metric that adds together all of the CLIFFORD candidates and all the Marcel candidates, then this new metric would be much, much better than Marcel. You haven’t failed at all.

    To be more elegant about it, of course, you’ll want to see how Marcel is identifying these candidates and incorporate it into your own metric. Or, perhaps more accurately, projection systems like Marcel will eventually want to incorporate your new findings, I imagine.

    Comment by cass — January 23, 2013 @ 10:38 am

  6. @Matt and @cass: I totally agree–for this I wanted to make sure I reported out on one type of failure (re: the hypothesis that relative risk would be better with CLIFFORD than Marcel), but there is likely something to be had by combining the two together.

    Comment by Bill Petti — January 23, 2013 @ 10:41 am

  7. Am I missing something or did you leave age related decline out of this? Pretty sure Marcel uses age as a factor of decline.

    This leads me to an assumption:
    I think the marcel candidates are strongly correlated to age, while it seems that you found skillsets that are prone to decline.

    This is where the difference in your candidates comes from IMHO.

    could you check this? and correct me if I’m wrong

    Comment by AC_Butcha_AC — January 23, 2013 @ 10:44 am

  8. We definitely need to see a follow-up article at the end of the season to see how well CLIFFORD and Marcel decline projections fared.

    Comment by MDL — January 23, 2013 @ 10:55 am

  9. Interesting stuff!

    Seems that you’ve jumped the gun twice.

    A lot of your CLIFFORD factors seem common sense as well.

    In terms of future progress:

    – Did you re-run the relative risk including the 5th parameter, change in K%? That might boost your relative risk even higher. It’s probably correlated with contact% but unless that correlation is perfect you’re likely losing some predictive power.
    – Have you considered including more years in your data set?
    – I’m somewhat new to Sabermetrics so perhaps this has been answered before, but is there a reason you’ve not built a linear regression model with these data? Wouldn’t this help you refine your model even further?

    Thanks! Not a failure at all.

    Comment by Jaker — January 23, 2013 @ 11:09 am

  10. out of curiousity, who were the three overlap players?

    Comment by juan pierres mustache — January 23, 2013 @ 11:22 am

  11. Are the Marcel candidates generally guys who regress back to their norms after a career year? From the way I understand Marcel to work, that would pretty much have to be it.

    Because if CLIFFORD finds a way to identify players who are suddenly going to drop off from their career norms, that would be interesting.

    Comment by matt w — January 23, 2013 @ 11:27 am

  12. I was just going to ask this. If an age adjustment wasn’t made, that right there could be the culprit.

    Comment by David Wiers — January 23, 2013 @ 11:46 am

  13. No adjustments were made in CLIFFORD. So Marcel is likely picking up the “natural” decline candidates given aging curves, etc, whereas CLIFFORD is identifying candidates based on other changes to their peripherals.

    Comment by Bill Petti — January 23, 2013 @ 11:53 am

  14. @juan pierres mustache Miguel Olivio in 2009, Robinson Cano in 2011, Ryan Zimmerman in 2011–they were both predicted, but not all actually declined by more than .030.

    Comment by Bill Petti — January 23, 2013 @ 12:01 pm

  15. i wouldn’t give up just yet either. I would want to know A) of the differences between clifford and marcel, which has a higher success rate? B) if clifford isn’t taking age factors into account, do you surpass marcel once you do add these factors in? you use UBR and SPD to account for aging but i’m not sure that’s sufficient unless changes in those stats from year to year are largely attributable exclusively to age. C) what happens if you expand the categories?

    If you feel like Marcel effectively disproved your hypothesis and that this research doesn’t warrant further pursuit, i don’t think you proved that in this article.

    I love the “inside baseball analytics” stuff, but i see this more as “showing your work” than “acknowledging a complete bust.” Maybe this could be part 1 in a series about developing a better prediction? Make some adjustments and report back, etc. Even if clifford doesn’t pan out, it could be used to test against existing projection systems and compare how, say, marcel does at predicting decline vs other systems

    Comment by tylersnotes — January 23, 2013 @ 1:41 pm

  16. And i’m sure Bill, that your peripherals in question are often associated with the “natural” declines/trends per Marcel’s system – i mean even the KPI’s you referenced: z-contact, UBR, speed, & FA/catching up with the FA, etc. all usually decline with age (at least that’s a good assumption). I think if you can disassociate variables with age trends, your research would have significant utility. I think it still has great significance because even accounting for age, a combination of certain variables will associate moreso with a visible decline.

    With that said, your system should still show value if it points to red flags that dont often correlate significantly with age trends, but it’s incestual if the relative risks you start with all relate to age.

    I always enjoy when someone works relentlessly to wind up going against their own hypothesis. It shows integrity.

    Comment by rotobanter — January 23, 2013 @ 3:13 pm

  17. Gotcha, thanks!

    Comment by David Wiers — January 23, 2013 @ 3:14 pm

  18. It seems like even if your decline predictions had already been found by Marcel, that doesn’t make your result a negative one. It still would have been a positive result (you confirmed your hypothesis), it’s merely that you would have replicated the positive result found by Marcel.

    Comment by Moonraker — January 23, 2013 @ 3:15 pm

  19. For starters, we don’t do enough of this. By we, I mean any researcher in just about any discipline–not just baseball analysis. More and more, journals, newspapers, online forums, and conferences are filled with the reporting out of positive results (“hey, look, I confirmed my hypothesis! I discovered a new thing!”). And while positive results are interesting and important in their own right, negative results are (or, should be) just as important. Any discipline progresses in large part by falsifying hypotheses, replicating results, and figuring out what doesn’t work so it can focus on what might.

    This is called “Reporting bias” and is a huge problem in all fields that I am familiar with. Boring negative results don’t get reported and when they do, they don’t get the press releases or the headlines in the lay press that the sexy positive ones do. You could have 14 people try to prove a hypothesis and fail, then one “proves” it correct with a p value of 0.1 and guess which one people remember? Now you have to do the original 14 studies over again as “confirmatory” studies and then they get reported. It’s a waste of resources.

    Comment by MikeS — January 23, 2013 @ 3:26 pm

  20. It would also be interesting to see decline in the previous year compared to career average..

    Comment by Jaker — January 23, 2013 @ 4:10 pm

  21. “I have not failed. I’ve just found 10,000 ways that won’t work.” -Thomas Edison

    Comment by GordieDougie — January 23, 2013 @ 5:34 pm

  22. Yeah, I would think Marcel is just picking the low-hanging fruit (predicting regression for guys coming off ridiculous years), whereas your system clearly is not doing so, but rather is filtering for guys who are coming off a relatively bad year. Look forward to more info.

    Comment by evo34 — January 23, 2013 @ 7:56 pm

Sorry, the comment form is closed at this time.

Close this window.

0.252 Powered by WordPress