What I Learned From Jeff Sullivan Learning About WAR

I read two articles the other day that helped clarify my thinking. For at least a year, I’ve been bothered by fantasy baseball’s obsession with rankings – especially preseason rankings. Imagine: here’s a list of the top 25 first baseman ranked in order of value. Except there isn’t much difference between the fourth best and the eighth best. After the 11th best it gets really ugly. Perhaps we should introduce some tiers, we’ll call them Great, Good, Average, Bad. But now there are tiers within my tiers and hidden tiers than span between my arbitrary tiers, etc. etc.

It’s annoying. It annoys me.

Fantasy Pros asked me to submit expert rankings a couple years back, which I dutifully provided in exchange for maybe a couple Twitter followers (ahem, @BaseballATeam). If memory serves I used something like 14 tiers to rank 25-30 second basemen. My next task was to take my position rankings and turn them into one big Top 300 list. I found myself wanting to change my tiers and trying to justify all the logical inconsistencies that arose from that process. That was the problem: process. Or more specifically, ranking players is a bad process.

I mentioned that I read two articles. Good for me. The first was here at FanGraphs in which Jeff Sullivan waxed poetic about the most important thing he’s learned from WAR. Basically, that teams are teams. The Red Sox didn’t win the World Series last year because David Ortiz mashed. There are 25 players on the roster, more are used throughout the year, and each part is only a small portion of the whole. If we replace any one player with another, the team is still pretty much the same. Maybe that’s less true if I replace Mike Trout with Scott Podsednik, but rules of thumb do tend to break down at the extremes.

The second article came my way via the post I wrote on the endowment effect. I’ve always been fascinated by Behavioral Economics and how it applies to everything I do, which includes a lot of fantasy sports. Apparently, reader/writer/commenter Jeff Quinton is also a fan of behavioral strategy. He presents an eloquent equation, which I’ve doctored ever so slightly for simplicity.

Fantasy Results = Analysis + Process

Most fantasy writing is geared towards analysis. As Jeff presents, via some very smart people (just read his article, it’s short and the quotes are golden), in typical business settings, Process is six times more important than Analysis. This was an A-Ha! moment for me. It clicked that rankings weren’t just an arbitrary waste of time, when used by themselves, they’re actually a bad process.

There are bad processes and there are bad processes. In the barely distant past, doctors, nobles, and city dwellers in general used to believe in a concept called miasma. Basically, the vapours and other foul stenches of a city caused disease. If you don’t breathe the vapors, you won’t get sick. And if you do get sick anyway, well you must have breathed the vapors.

Public health students will confirm that John Snow is credited with debunking the theory of miasma. This was all before he joined the Night’s Watch. That advance led to germ theory and other modern medical breakthroughs. Belief in miasma is kind of like using fantasy rankings. If you follow the theory behind miasma, you’ll leave yourself prone to illness from bad drinking water, food, and diseases passed by touch, cough, blood, etc. But you’ll also follow your doctor’s advice when he says it’s time to go breathe the health air of the countryside. Believing in miasma is better than believing that disease is entirely random or caused by a divine curse. Miasma can improve health outcomes even if germ theory is far superior.

So without further ado, I present the unified theorem of Perfect Fantasy Process (PFP, also known as Pitcher’s Fielding Practice).

Unfortunately, such a thing doesn’t exist, at least not in the sense that it can be easily grasped or understood. There are different processes to use for the draft, managing your team, making trades, using the waiver wire, and more. There are competing strategies, all with merits. If when playing chess you always use the Sicilian Defense, you will be countered with the Smith-Morra Gambit (this is just gibberish that I Googled). The same goes for fantasy baseball, if you use a certain strategy and your opponent recognizes it, they will counter.

I’m most known for my frequent waiver wire streaming. Sometimes, a rival will attempt to replicate my strategy. That’s bad for me because a league’s waiver wire can usually only support one to three streaming owners. More than that and the streamers will generally do more harm than good. So rather than wake up at 4:00 AM so that I can make my moves first (people do this and it’s unholy), I just switch to a strategy where I hold my assets. That usually requires a subtle restructuring of my roster, but I can do that over time as opportunity allows. One thing about streaming owners is that they dump decent players in their quest for another mediocre pitching match-up or four more plate appearances. They’ll hand you useful role players for nothing, you just have to wait until they give up the right one for your roster.

That flexibility is the skill that we need to be practicing and improving. We need to learn the alternatives available to us when our rivals present a certain strategy. We commonly extol the virtues of zigging when others zag, but it requires a depth of knowledge to recognize when others are zagging and to know how to zig. When drafting, we shouldn’t just draft the best player available, we also need to pick players who improve our team’s depth and aid our ability to compete in every category. I know a lot of my rivals are just bidding for the player. They want Sonny Gray because they read that he’s a sleeper, so they’re willing to bid up to $12 for him. Maybe they aren’t paying attention to the giant hole at second base that gets worse and worse with every guy off the board. Gray may be a fantastic sleeper, but those $12 might be more usefully invested on a non-sleeping second baseman.

Rankings do have their uses. Owners need some way to organize a large quantity of data. I build league specific price sheets so that I know roughly what a player is worth. But I’ll adjust those prices during the draft to account for inflation and the categories that I need most. If my team is short on steals and I have one outfield slot left, I’ll pony up extra for Michael Bourn. The point is, do your analysis, build your rankings, and then put better processes in place. Don’t stop at rankings.




Print This Post

Brad is a former collegiate player who writes for FanGraphs, MLB Trade Rumors, The Hardball Times, RotoWorld, and The Fake Baseball. He's also the lead MLB editor for RotoBaller. Follow him on Twitter @BaseballATeam or email him here.


25 Responses to “What I Learned From Jeff Sullivan Learning About WAR”

You can follow any responses to this entry through the RSS 2.0 feed.
  1. Bill says:

    Good read. Would love to know more about how to properly adjust rankings and price points during a draft.

    Vote -1 Vote +1

    • tylersnotes says:

      One thing I do is set targets for my team at each category (or point targets in pt leagues) and use this as the foundation during drafts/auctions. Have a spreadsheet with your projections, as you draft fill those stats in against your targets. By mid draft you should know where you’re lacking and be prepared to spend more in those areas

      Vote -1 Vote +1

      • Bill says:

        Would seem to be less valuable in points leaves right? I’m looking to do for both roto and points.

        Vote -1 Vote +1

      • Brad Johnson says:

        For points leagues, it’s just a function of balancing points above replacement (PAR) with position scarcity. Usually, that means take the player with the highest PAR. Sometimes, budget or draft order might make it optimal to take a 100 PAR SS over a 200 PAR SP because the next tier down at those positions might be -50 PAR and 125 PAR respectively.

        Vote -1 Vote +1

      • bmiltenberg says:

        Thanks. What’s the easiest way to calculate replacement level? If it’s a 12-team points league, is replacement level the average of, say, the 11-14 shortstops ?

        Vote -1 Vote +1

      • Brad Johnson says:

        I usually do number of roster spots plus some over rostering. So if there’s no MI in a 12 team league, then I’d look at around the 15-17th SS as my replacement level.

        If there is MI, then probably about the 25-27th SS.

        Vote -1 Vote +1

      • GilaMonster says:

        I break them into tiers. Calculate the drop off in production between tier 2 and tier 3…etc. I then target a few guys that provide the most value.

        During the draft I compare what I have to what everyone else does. If I have HR and RBI, I might target the R/SB guy to fill the hole instead of a big bad.

        Vote -1 Vote +1

      • tylersnotes says:

        search the fangraphs archives for Zach Saunders’ fVARz (fantasy value above replacement) series; it’s a great primer in figuring out replacement level. Read the comments, especially for the update. Saunders’ model has some flaws and the comments help you figure out how to adjust to your needs.

        Vote -1 Vote +1

      • bill says:

        thanks guys

        Vote -1 Vote +1

  2. tylersnotes says:

    Loved this piece. this has been a stellar day for reading rotographs. I love the influx of articles geared toward process and “big idea” pieces in addition to the individual analysis. It seems to me rotographs has made huge improvements this year.

    Vote -1 Vote +1

  3. The Stranger says:

    The idea of process is especially important as more and more scoring systems proliferate through the fantasy world. I play in three leagues; a 6×6 H2H league, a 5×5 roto league, and an ottoneu points league. Each of those has different roster sizes, positional requirements, and waiver rules. And that’s only scratching the surface of the rulesets I’ve heard of. Which is to say, rankings aren’t getting you very far; they’re by far the least useful fantasy content out there.

    Unfortunately, there are so many variables at work that it’s hard to tell how to really win a league. Most of us have a sample size of 1-5 leagues per year, which isn’t enough to tell us much of anything. Which is why I’m once again going to half-jokingly suggest RotoGraphsGraphs so we can aggregate fantasy season data and break these things down.

    Vote -1 Vote +1

  4. Jeff says:

    Brad- Glad my article could be of help. The strategy articles have been great.

    Vote -1 Vote +1

  5. pudieron89 says:

    What I Learned From Brad Johnson Learning From Jeff Sullivan Learning About WAR:

    Vote -1 Vote +1

  6. My echo and bunnymen says:

    Bloody brilliant. This will help me further improve my own thought process on my own thought process.

    Vote -1 Vote +1

  7. Jason says:

    Fantasy baseball needs to develop cool shorthand for the various strategies. “Waiver Wire Streaming” just lacks the cachet of “Sicilian Defense”.

    Vote -1 Vote +1

  8. tz says:

    Your second-to-last paragraph is the great takeaway here.

    As an analogy, if you play poker, even if you KNEW what was in everyone’s hand, there are elements of live strategy that can quickly override any theoretically-correct “best” option. If this weren’t true, there wouldn’t be anyone playing poker professionally.

    The zigging and sagging you mention is the key component for that gray area between the opening rounds and the endgame.

    Vote -1 Vote +1

  9. Fatbot says:

    A better equation is probably fantasy results = analysis + process + luck. Fantasy is a game like poker where everything an owner does hopefully adds up over time to minimize the random factor so that the analysis and process part (what people like to think is “skill”) wins in the end. The people that put in the time to gather the knowledge and make the informed moves over time will win more, but no matter how much analysis and process tries to drive it out, that luck factor still remains.

    Vote -1 Vote +1

    • Jeff Quinton says:

      Luck is definitely a factor, but because it is a constant I left it out.

      Vote -1 Vote +1

      • Brad Johnson says:

        I don’t agree that luck is constant. Improvements in analysis and process can decrease the influence of luck on the equation. Last season, most people rostered Donaldson through sheer luck, I have a formula I use on spring training data that identified him as a breakout candidate (along with Brown, Belt, Smoak, and Cain. So it went 3-for-5). So for me, Donaldson was an analysis win.

        A few years back when Ben Zobrist broke out, I just happened to have him on my roster filling in for an injured Jose Reyes. That was luck.

        Similarly, waiver wire streaming can be seen as a process of trading the opportunity for long term, high variance luck for short term, low variance luck. There will be 10-20 players you wish you didn’t cut at the end of the season, but there will also be lots of little bits of great performance. By ensuring more PA and IP, you are also reducing the necessity for luck.

        Vote -1 Vote +1

  10. Jeff Quinton says:

    I agree that how much variability (call it luck) you choose to take on is not constant. That is a product of process and analysis, but that is already a part of the equation. I guess, put differently, how much variability you choose to take on is not a part of luck, but rather a part of decision making.

    Once you have chosen how much variability you are going to take on, then that variability is set and the results will be out of your control.

    Vote -1 Vote +1

  11. Gary Mugford says:

    I did pretty well over the years in a snake draft/points league with a strategy that basically worked along the lines of calculating expected points and then a draft spot advantage number. The DSA was a gap score between any one player and a player some number of rankings below him. The decision on how many players was a function of how many teams there were (it varied). I finally settled on two-thirds of the number of teams rounded down. So, in an average league of 10 teams, I would gap six players.

    What that would do was to account for positional scarcity AND potential drafting runs, without ignoring the total points. That led to drafting the counting catchers earlier then higher-scoring corner guys, because one medium corner guy and the top catcher ended the season outscoring one star corner and a ninth hole type catcher.

    The DSA more or less automatically sorted everybody into tiers and, if adhered to completely, eliminated personal bias. The DSA anticipated positional runs. It wasn’t completely run-proof. If a top catcher had the top DSA and was drafted accordingly, with seven or eight really good corner guys, still available, for example, they might all be gone before the draft order got back around to you. But such excessive runs are rare. Or were rare.

    We all have favourite teams, favourite players. But come draft day, we are there to win. I always took the top DSA player on my list, mindful of completing my roster. Over the years, in our league, I mostly finished first if healthy, second if not.

    Maybe it wasn’t the strongest league (it was not), but the DSA tool proved very effective within it.

    Vote -1 Vote +1

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>