If Elias Used WAR

Yesterday, Buster Olney reported that Major League baseball and the Major League Baseball Players Association were nearing agreement on the new CBA. Olney notes that the new collective bargaining agreement will likely address free agent compensation by getting rid of compensation picks in the first round. As I noted last week, there are many fundamental problems with free agent compensation, as we currently know it. My focus last week was on how the system unintentionally provided incentives for rich teams to not just sign one type A free agent, but sign multiple. In addition to this unintended byproduct of the system, I briefly mentioned the archaic stats used to rank type A and B players. This led me to wonder what would happen if MLB instructed the Elias Sports Bureau to use more advanced metrics.

The rankings are based off of the past two seasons’ worth of statistics. With that in mind, I began thinking about what an idealized ranking system would look like. It didn’t take long to figure that Wins Above Replacement would be much better than the current equation used by the Bureau. Under the current system all players are divided into five different groups:

Groups Stats (From MLBtraderumors.com)
1B, OF and DH PA, AVG, OBP, HR, RBI
2B, 3B and SS PA, AVG, OBP, HR, RBI, Fielding%, TCs
C PA, AVG, OBP, HR, RBI, Fielding%, Assists
SP Total games (total starts + 0.5 * total relief appearances), IP, Wins, W-L Percentage, ERA, Strikeouts
RP Total games (total relief appearances + 2 * total starts), IP (weighted slightly less than other categories), Wins + Saves, IP/H ratio, K/BB, ERA

Once the players have been divided, they are ranked against their peers. The ranking works as follows: within each statistic, the player in question is ranked against his positional peers. If there are 30 players in a group, then the player the most HRs gets credited with 30 points, the player with the second most HRs would be credited with 29 points and so on until you reach the player with the fewest HRs, who will be credited with 1 point. All points are then scaled so that the max score for each stat is 100 (divide by the number of players in the given group and them multiply by 100). The scores for each stat are averaged to give the player his final score. If a player led his group in all stats, his final score would be 100. Once each player has been given a score, all the groups are aggregated. The top 20% of the aggregated list is deemed a type A player and 21%-40% is deemed a type B player.

Thanks to MLB Trade Rumors I was able to grab the rankings of all players for the 2010-11 seasons and take a look at their respective WARs. The first thing I did was divide the players into their respective groups. After the players were in the right groups I looked at the composition of each group. The average two-year WAR for type A Catchers was 6.2. The average type A player in Group 1 (1B, OF, and DH) had 8.1 Wins Above Replacement over the past two years. Similarly, Group 2 (2B, 3B and SS) had 8.1 Wins Above Replacement over the same time span. This seemed to be an interesting coincidence, but then I looked at Group 4 (SP). The average type A Starting Pitcher had 8.2 Wins Above Replacement over the last two years. It is surprising given their respective makeups, that groups 1, 2 and 4, would have nearly identical WAR. Finally, I looked at type A Relief Pitchers, and found that the averaged a measly 2-year WAR of 2.5. By looking at the average WAR for each group we can appreciate how type A players in certain groups are relatively valued. It seems as though all positions are valued approximately the same under the current system (catchers are slightly over valued) with the exception being relief pitchers. A type A relief pitcher nets the same compensation as any other position, but the value of the type A relief pitcher is on average a third of the value of other type A players (as measured by WAR). This overvaluation of Relief Pitchers is clearly an issue that needs to be addressed in the upcoming CBA.

Finally, I compared the current rankings as calculated by Elias and the idealized rankings using WAR where the top 20% of players according to WAR would be considered type A. The Venn Diagram below shows two sets and their intersection with each other. The red circle is the number of type A players according to the Elias ranking, and the blue circle is the number of type A players according to my WAR ranking. The intersection of the two groups, as seen in the purple shading, is the number of players that belonged to both sets, i.e. the number of type A players unchanged by the new ranking. At Bradley Woodrum’s behest, I’ve slapped in some festive Marios to make these impressive diagrams even more digestible.

 

If you look at the Venn diagrams above, you’ll see that a high percentage of the Elias type A players would continue to by type A players under the WAR ranking.  If you exclude RPs, then 92% of Elias type A players would be WAR type A players. This is a much higher number than I would have expected. That being said, you can see from the numbers that lie only in the blue areas that the Elias ranking misses a lot of good players – has a high type II error. Finally, the Relief Pitcher Venn diagram illustrates the huge incongruence of the WAR rankings and the Elias rankings. Of the 52 type A RPs, only one of them would be type A according to WAR. I think the WAR ranking would be a great improvement, but the Elias rankings was not as awful as I had initially thought.



Print This Post





Sort by:   newest | oldest | most voted
Yirmiyahu
Member
Yirmiyahu
4 years 10 months ago

I suggested this in the comments to your piece last week, but it’s more on-topic here. If we’re going to use some system to rank free agents for compensation purposes [separate debate], why do we even need to use statistics? Why not just use his actual market value?

That is, something like: a guy signs a contract over $30M and he becomes a Type A, between $10M and $30M and he becomes a Type B, and between $5M and $10M and he becomes a Type C.

Or alternatively, come opening day, rank all of the offseason’s free agent signings according to the size of the contract, and make the top 15 guys Type A’s, the next 15 Type B’s, and the next 15 Type C’s.

I arbitrarily picked the numbers, and you can adjust them so that the appropriate number of players fall into each box. Or come up with any number of variations.

But such a system would eliminate the disconnect between a player’s ranking and his actual market value. The result would be that teams couldn’t game the system to collect picks, and individual players wouldn’t have their salary artificially deflated because of their ranking.

Food For Thought
Guest
Food For Thought
4 years 10 months ago

Where does the AAV vs total amount trade-off come into play with this system? Do we just average out the contract total and ignore what the player is actually getting paid that year? What about team/player/vesting options?

I like this idea in theory, but there are going to be ways to pay players but game the system (just like with taxes!).

Yirmiyahu
Member
Yirmiyahu
4 years 10 months ago

I say total guaranteed contract. You could do total guaranteed value divided by total guaranteed years (i.e., AAV), but I think using AAV would overvalue certain older players who are good but seen as too risky to give long-term contracts to (like Oswalt and Ortiz this year).

As far as options and incentives, it’d be a minor problem if teams tried to use them to game the system, but I think that issue infinitely pales in comparison to the problem of using the Elias system or WAR or any other statistical-based system.

gnomez
Guest
gnomez
4 years 10 months ago

Disagree. Suddenly we’d see guaranteed contracts at league minimum and $15m incentives that kick in at 3 PA or 1 IP.

Brad Johnson
Guest
4 years 10 months ago

Gnomez,

The commissioners office approves all contracts. They would not approve something so blatantly designed to get around the rules.

Further Food For Thought
Guest
Further Food For Thought
4 years 10 months ago

I suppose the system will never be able to perfectly evaluate players (if so, give us that stat Elias!), but the money system is just too easy to game. At least with some measure of past player performance, you are not affecting a player’s value based on… well… his value. The system you proposed will dampen the high end salaries while raising the low end ones. That is NOT a direction you want a league to go (see: NBA).

Really I just don’t think that teams should be compensated for losing free agents, but that’s another story.

jwb
Guest
jwb
4 years 10 months ago

“$15m incentives that kick in at 3 PA or 1 IP”

Here’s a way around this problem. Look at the last N years of the player’s performance and count the fraction where the player has reached those incentives as a part of the AAV of the contract.

Let’s say a team signs Erik Bedard to an incentive laden contract which pays a $1M bonus at 75 IP and $1M for every 25 additional IPs up to 200. Let’s arbitrarily pick N = 3. In the last three season, Bedard has pitched 83, 0, and 129 innings. This system would give him a 2/3 chance to reach 75 IP, a 1/3 chance to reach 100 IPs and 125 IPs, and no chance to reach 150 or more IPs. So add $666,667, $333,333, and $333,333 to the guaranteed AAV of the contract.

Hark
Guest
Hark
4 years 10 months ago

I am the Yankees. I want to tank another team. I pay some scrub $30 million because I can afford it. They are treated as Type A. I sit them on the bench all season to play someone else that I bought for less.

This is why such a system is kind of dumb.

Ian R.
Guest
Ian R.
4 years 10 months ago

Wait, what? First of all, if the Yankees did this, they’d be giving the other team a draft pick. How does that qualify as “tanking?” Second of all, even the Yankees don’t have unlimited money. They’re not going to pay a scrub some absurd amount of money just to spite another team.

There are ways to cheat the proposed system. This isn’t one of them.

cs3
Member
cs3
4 years 10 months ago

Total max contract value, including ALL incentives + options would be the best way to go if this system was used.
But if you do this then there would be tons of contracts given out to players just below whatever threshold is established for each level. And those thresholds would then artificially impact the market rate for players, which is what were trying to avoid in the first place.

No matter how you compensate/punish teams for signing FA’s, there will always be drawbacks and ways to manipulate the system

tigerdog1
Guest
tigerdog1
4 years 10 months ago

I have been preaching that Elias should use money as the measuring stick to determine compensation. Actually, you wouldn’t need Elias, or much of any formula. Use the average annual value of the contract that the free agent player signs with his new club. After all, that IS his value, and that is the reason that his former club should be compensated for losing the player. They’ve lost a player worth X amount of money, so they should receive a corresponding amount in compensation, though not full value in raw dollar terms.

If a Type A player is defined as one in the top 20%, then take the top 20% of MLB salaries, and any player making above that amount is a Type A free agent. If his club offers arbitration and he signs elsewhere, the club gets a draft pick in Round A- or two picks, however the formula works. Position doesn’t matter. If the market values players at certain positions higher than others, that will be reflected in the rankings.

Eric
Guest
Eric
4 years 10 months ago

Thank you Mario! But our Type A Free Agents are in Another Castle!

reillocity
Guest
reillocity
4 years 10 months ago

Out of curiosity, which 2 C, 4 1B/OF/DH, 2 2B/3B/SS, and 2 SP are Type A per Elias but not per your WAR criteria?

Jason
Guest
Jason
4 years 10 months ago

I third the call to rank players according to the value of their contracts.

There is no reason to replace one flawed statistical evaluation with another flawed statistical evaluation. WAR obviously overrates defense and underrates many relief pitchers relative to the market. The fact that according to WAR many teams overpay relief pitchers does not change the fact that your team lost a relief pitcher to a team willing to pay him a lot of money.

Bradley Woodrum
Member
Member
4 years 10 months ago

Holy #%&@ those Venn Diagrams are awesome!

AndyS
Guest
AndyS
4 years 10 months ago

I am loving the use of Mario sprites on this site recently.

adohaj
Guest
adohaj
4 years 10 months ago

Who was the 1 relief pitcher?

AK707
Member
AK707
4 years 10 months ago

“I think the WAR ranking would be a great improvement, but the Elias rankings was not as awful as I had initially thought.”

This understates the degree that the current system overvalues relievers. None of them should ever be part of a pick compensation system.

making money the measuring stick
Guest
making money the measuring stick
4 years 10 months ago

The Rays will never get another comp pick again.

fhomess
Guest
fhomess
4 years 10 months ago

I don’t understand why the total ranked players is different between Elias and FanGraphs. For example, why are there 26 more SP’s in the exclusively blue section than in the exclusively red section. Shouldn’t the total number of players ranked be the same?

The only explanation I can think of is that 26 more FA SP’s are ranked in the top 20% of all SP’s, but that seems like a large enough number to question whether or not the qualifications for the two lists are the same.

DCN
Guest
DCN
4 years 10 months ago

Is it possible that good defensive catchers are undervalued by WAR? It seems like there’s more room for hidden value at that position than anywhere else, because their defensive role is so complicated and because of their effect on the pitching staff. But that’s not really a problem with the article, just something I thought about when he said catchers were overvalued.

tim
Guest
tim
4 years 10 months ago

My biggest problem with WAR is that I have no idea how it’s computed, and I seriously doubt you can have one stat measure everything. It’s kinda like the NFL’s quarterback rating, but at least they show out that’s computed on the NFL website. And once you learn how to compute the quarterback rating, you realize you don’t need it. It basically mirrors touchdown-to-interception rate. What’s more, you can actually compare a team’s winning percentage with a certain player as opposed to without him, and figure out an actual “wins above replacement” rate without the complicated formula.

wpDiscuz