Great start to the evaluation. Hopefully, this preamble will remove a lot of the angst around the rankings. Thinking through the comments from last year, you appear to have addressed some of the most common concerns.
The only big item missing is whether we are evaluating teams for this year, the next 3 years, or sustained success over a longer period (like 10 years). Obviously the evaluation is dramatically different if we are looking primarily at 2011 results or trying to see which franchise is best positioned to be dominant through 2020.
I also completely agree that financial considerations should be the most important weight. Nobody likes to admit it but your probability for success is tied closer to your teams checking account and revenue streams than it is to your talent evaluation or GM’s trading ability. That sucks but it’s true.
I know it’s a dissatisfying answer to a lot of people, but the reality is that these rankings take both the present and future into account. 2011 matters, but so does 2012, 2013, and so on. The current year matters the most, as the strong weighting for present talent shows, but teams that have given up some future value to try and win now will be harmed in the rankings by that decision.
This is not an estimate of how likely each team is to win the World Series this year. It is much more of an exercise to determine how healthy each organization is going forward. We’re attempting to discern which organizations are setup to contend both now and in the future.
How are you ensuring cross-rater consistency?
E.g. How do you ensure that 9outof10 in finances equals the same thing across raters and teams?
Wouldn’t it have been better to have one person do all the Present Talent ratings, another person do Financial Resources and so on?
I found it interesting going back into 2010, and looking at MLB payrolls, and while their payroll doesn’t necessarily reflect financial health of an organization, I still found it interesting that only 3 of the top 10 payrolls in MLB last year made the playoffs.
The Giants were 10th, and are a good example of a team that may have a high payroll but not exactly be in the best financial situation, with their park, and the fact that Barry Zito is almost 20% of their payroll.
Obviously financial well being and revenue streams can put you at an advantage, but its the use of the resources that seem important to me. I mean, what good is a lot of resources if you’re wasting them with poor personnel and fiscal decisions?
Part of the team’s strength is how it generates revenue. If a team in a small market generates strong revenue, that tell us good things about the team.
I’m a bit concerned you might be double-counting market-size, though. To avoid that, I’d want to adjust financial resources for market size (not unlike DMZ’s revenue-sharing plan), so teams would only get added financial resource credit for the extent to which they out-perform their market.
After all, the advantages of the bigger market are already being counted in the Present Talent (and to a lesser degree Future Talent) section.
It’s hard to call AA better than three established guys like Friedman, Epstein, and Cashman just because he magically disappeared Vernon Wells and got solid value back for Roy Halladay. Say what you want about Cashman’s financial muscle, but he’s done exactly what he’s supposed to do with that money, get them to the postseason year in and year out. Epstein’s on a similar level except you get the feeling he could do exactly what Friedman’s done if given the same situation. And Friedman’s pretty unassailable these days.
AA looks good but when compared with 3 successful GM’s with long track records, he should be ranked 4th.
I’d venture to guess the Cubs will be much higher than 27th. They are around a 0.500 club with a top 5 payroll and about average (slightly above?) minor leagues. Their front office would drag them down, but that should wash with the financials – so I would put them around 15.
Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up Grow up
Comment by neuter_your_dogma — March 21, 2011 @ 2:42 pm
So you’re saying we shouldn’t expect the genius of Jack Z compared to the apparent dolt Frank Wren to propel the mariners leap the Braves in spite of every other possible, rational factor?
They don’t need to as long as the raters are generally held constant. I’d prefer if they didn’t have voters abstaining but if they legitimately believe that they can’t objectively evaluate something they’re close to, then I suppose it’s for the best.
I have to say, I find this methodology vastly improved over last season. Going with component scores seems like a very reasonable way of doing things. My only little thought is that we may want to at some point look at these components as combining non-linearly. I would think that these would have a relationship sort of like:
Organizational Health = (Present Talent + Future Talent + $$$) x Operations
Where there is some weighting on each of the assets terms, while the range of operations values determines its importance (i.e. max/min of [0.5,1.5] would be more influential than [0.9,1.1]).
I say this because the first three aspects can be considered interchangeable commodities. Present talent can be traded for future talent, vice versa, and money can be exchange for both (within limits). On the other hand, Operations seems to really be a limited multiplier on these. A good front office will steadily grow these assets, while a bad one will steadily lose value of assets.
So using something like that and using these weights, we’d probably be seeing a formula more like:
Value ~ (0.4*Present + 0.4*Money + 0.2*Future)*(0.875 + 0.25*Ops)
Though if we believe our own $ valuations of prospects, I do wonder if its better to use absolute values rather than normalized ones.
It’s not necessarily a dissatisfying answer, but it will certainly lead to a more subjective, qualitative conclusion. To me, this series will be less valuable and interesting than if you were to precisely define “organizational success”, as that is the entire goal of the rankings.
When you start out with a wishy washy question, you end up with a wishy washy answer.
Does future talent include only prospects? non-25 man roster types? or is it the measurement of how much projection the entire organization’s current player personnel is viewed as having…
1. For instance, stating the current ability of Pedro Alvarez in the “current talent section”, but accounting his talent projection in the “future talent section”.
2. Accounting for only the Allie/Taillon types/projections in “future talent”… Guys who haven’t quite influenced the actual Pirates MLBallclub yet…
From last week: “While the weights that you guys vote on may not match exactly what we end up using for the model, the results of the polls below will be a significant factor in deciding how each category is valued.”
I’d just like to say, not all Braves’ fans are this self-absorbed so as to pettily cling to perceived slights pertaining to things that absolutely do not matter.
Of course, given how much outrage I’ve seen over a projected line-up that involves Heyward batting 6th instead of 2nd, I can’t exactly say we always have a good grasp for what matters and what really does not.
Indeed, Telo; and a series evaluating American politicians on the basis of which would make the best president would be more valuable and interesting if you were to precisely define what qualifies as success for a president. Which comment will probably have some yelling “Politics!” — which is the point, because the reason for that reaction is that we don’t all *agree* in that regard: we have different definitions of success based on the different policies we prefer. Similarly, there is no precise, universally-accepted definition of organizational success. You just do the best you can amidst the differences of opinion.
Comment by The Ancient Mariner — March 21, 2011 @ 4:18 pm
Er, sorry, should have refreshed before I posted. But by this measure, future talent may have more to do with players on the current roster than prospects (i.e. it’s not just “the team’s minor league system;” it also includes all the currently-signed building blocks of future rosters). And if that’s the case, shouldn’t it receive more weight? Longoria’s years after 2011 are way more valuable to the Rays than his 2011 year. If we’re looking at the overall strength of an organization, the contributions it can be projected to receive from players in 2011 doesn’t seem any more important than the contributions it can be projected to receive from players + prospects in 2012 and beyond.
Just so we are clear, those are my opinions about your predictions of what fangraphs writers’ opinions are.
Comment by Steven Ellingson — March 21, 2011 @ 4:20 pm
“If *they* legitimately believe”? It has nothing to do with what they believe, or what anyone might legitimately believe; it has to do with the paranoid, self-righteous judgmentalism (and associated schadenfreude) exhibited by a lot of commenters here over the last year’s rankings.
Comment by The Ancient Mariner — March 21, 2011 @ 4:21 pm
I gotta say I really like this idea. I think something along these lines should be considered for next year’s evaluations.
Who cares if you outperform your market, but you’re still 27th in baseball in revenues? The important point for retaining the talent you developed, and importing established talent, is the 27th rate in revenue, not that your 27th when you should be 31st.
Overall financial resources is separate from present and future talent, in the sense that if you have good present and future talent with good revenues, you can more easily retain those going forward (See Adrian Gonzalez and Felix Hernandez).
Also, greater financial resources make it easier to improve poor present and future talent (If you have a bad player on a $12m/yr deal, at least when it’s over you’ll theoretically have a second chance to spend that $12m more wisely).
Comment by a seattle fan — March 21, 2011 @ 4:22 pm
fyi, I wrote this for the hardball times back in January. I’m definitely on the AA bandwagon. I just can’t see a successful argument that he’s a better GM than that other trio when we have such a small sample of performance to evaluate him on.
ok ok. I think what I’m getting at is that if you’re a writer for Fangraphs you should be pretty damn objective. As such, your ranking for the team you follow closest should actually be your most accurate ranking rather than simply a biased ranking.
Bothers me that the most important and least important factors are given approximately the same weight.
But, for this exercise, it’s probably a necessity. If we ranked Financial resources at 60-70%, it’s be a boring series, with little controversy.
I do think the first 3 released were very well done within the parameters fo the activity.
Links to season’s previews and minor league talent have been very useful.
I wasn;t the biggest fan of the series, given the weightings of the categories, but I have enjoyed (and found useful) the first 3 released.
Comment by CircleChange11 — March 21, 2011 @ 5:14 pm
Great job on the new format, Dave.
What I like is that it’s laid out a lot more clearly how you’re ranking the teams, and the numbers make it easy for anyone to use your evaluations to generate custom rankings. If you disagree with the weights, it’s easy enough to plug the numbers into a spreadsheet with different weights and come up with a different overall ranking of teams without losing the fundamental ratings. Or, if you disagree with the assessment of a particular team’s management, you can see where the team would fit with your view of that team’s baseball ops. It’s nice and modular, so even if you disagree with certain judgment calls on the specifics, it doesn’t make the rankings useless to you.
I have no idea what the point of your comment is, besides to deflect heat aimed at Cameron. As usual.
What I think you are trying to say is: If you were to define “Organizational Success” from the start, folks would argue about that criteria before ever getting to the debate of the actual rankings. This is… illogical, to put it kindly.
SOME GUYS: Hi everyone, we’re going to do a ranking of X, based on the criteria of Y.
OTHER GUYS: Sounds great, I love talking about X. But what exactly is Y?
SOME GUYS: Well, that’s the tricky part. We’re not exactly sure. We know what Y is qualitatively, but our individual opinions on EXACTLY what Y is varies (and we’re sure you have your own opinions on Y, too). And while it would it would be pretty easy to just decide on a set of Y, right or wrong, agreeable or disagreeable, we’re each just going go ahead without our own, vague versions of Y to come up with the rankings. Shouldn’t be a problem.
OTHER GUYS: Well…how are we supposed to judge whether we agree with your rankings, if we don’t know the exact criteria you used when creating them?
If the purpose is “how healthy the organization” is going forward as Dave said, then I don’t think it’s double counting, because having more financial resources because of a big market doesn’t just show up in current or future talent – it’s an advantage that the organization will always have. And if a big market team spends their payroll poorly they may not even have good current or future talent but they will still have that financial muscle to flex in the future, but your proposal wouldn’t reflect that upside.
However, if the purpose if the rankings is to understand what organization best outperforms their expected level of performance, then yes, I would agree that you’d only want to give credit for outperforming market size.
The Giants are actually in great financial shape and their ballpark is a big reason. Yes, they are paying a mortgage on it, but that mortgage will be paid off in the very near future giving them a nice windfall. The ballpark also creates enormous opportunity for revenue streams that they can keep off the books of the baseball franchise. Other factors:
1. Their ownership groups is very well heeled and not dependent on the financial well being of any one owner such as, say, the Dodgers or Mets.
2. They are in the 5’th largest market in the country.
3. They claim as their territory the Silicon Valley area with a bunch of very wealthy companies and their CEO’s and employees. If the A’s want to move to San Jose they wil need to pay a very hefty territory fee.
4. They increased their payroll to about $114 M this offseason so Zito’s salary is actually less than 15%.
5. They have the windfall of winning the WS, which admittedly is a one time infusion of money, but that led to their season being already essentially sold out for 2011.
6. They have always had a national following, perennially leading the majors in road attendance and perennially being one of the top teams for the sale of merchandise. That has also exploded since winning the WS.
Dave said above: “I reduced the weighting for financial resources from what I had originally targeted it as.” I agree with his original inclination. I would have made financial resources more like 35-50% since market size is the only variable that is relatively static and for the most part can’t be affected. Your current talent, future talent, and front office all can be replaced if they aren’t working (albeit it takes some time).
The most confusing part of that post is that Rogers is ranked 5th worst ownership in the division? Wha–? How is Angelos any better? Especially when Rogers has taken major strides in improving talent both on and off the field (Anthopolous, more scouting, international signings) and seem committed to winning.
I don’t have a problem with AA being 4th on that list, simply because being the 4th-best GM in the AL East probably makes you, what, about the 4th best GM in baseball? Slightly facetious, but not entirely. The ESPN write-up basically acknowledges that, too. There’s an exceptional bunch of GMs in the East.
On the other hand, I disagree with ranking Rogers below the Steinbrenners, because, again as the author acknowledged, they’ve straightened out over the last couple years, and they tend to send the signals that they’re ready to spend when the time comes. I think a compelling case could be made to put the current Rogers regime directly behind John Henry in the rankings.
Do you feel that the weighting system employed here would result in an accurate ‘prediction’ if you were to use FG Organizational rankings from 3 years ago?
Comment by GhettoBear04 — March 22, 2011 @ 12:27 pm
It was like squeezing blood out of a turnip, but you finally defined what your rankings are intended to rank. Given that you are predicting success and not saying who’s doing a good job, you should have not backed down from a higher percentage for having money.
You still confuse the issue by calling this “organizational rankings” rather than “teams likely to succeed.”
I dunno, Theo’s been able to make some shrewd trades (although none as good as the Vernon Wells trade), and his scouting department has produced some outstanding young players, but what about his track record with free agents?
Mike Lowell riding the bench for $13M?
That’s $60M+ ill-spent dollars last year alone. AA’s saved the Jays more than that with one off-season trade.
Adam: J.D. Drew has been about as valuable as what he’s being paid, and everyone wanted Dice-K. You really can’t complain about either of their contracts. As for the other three players you mentioned, yeah, they’re bad contracts. But a bad contract for Boston (or New York) isn’t nearly as big a deal as for any other team. Boston can afford to offer Lackey a big contract and hope he returns to dominance. I’m not saying it was a good deal, and certainly not defending Lugo or Lowell’s signings. But as a big-market team, the biggest benefit is that you can afford to make a few mistakes.
Comment by skippyballer486 — March 23, 2011 @ 3:12 am
As a Cubs fan – I completely agree. The Operations multiplier should be negative for the Cubs. Even if they had the most money, the inefficiencies of the FO would devalue it.
As a Twins fan as well… They would have a higher multiple. Wisely developing talent, not throwing away money on bloated, stupid contracts (Alfonso Soriano).
This formula could really work. It might take a few itterations to get the exact formula, but it would be worthwile in my opinion.
Hi,No the pad does not require a Kernel modification, specifically because there are Two connection points for the internal USB system.My point was that connection to ONE of these points causes the Kernel to error, but connection to the OTHER USB point has no significant problems.
Comment by Leonora Schweinert — November 13, 2011 @ 3:43 pm