The State of Analytics Within MLB

This post was written by Adam Guttridge and David Ogren, the co-founders of NEIFI Analytics, an outfit which consults for Major League teams. Guttridge began his MLB career in 2005 as an intern with the Colorado Rockies, and most recently worked as Manager of Baseball of Research and Development for the Milwaukee Brewers until the summer of 2015, when he helped launch NEIFI. As part of their current project, they tweet from @NEIFIco, and maintain a blog at their site as well.

Much has been made lately of “title inflation”; the presence of multiple personnel above the role of assistant GM within MLB front offices. That’s a symptom of something larger, rather than a phenomenon unto itself. Front offices are growing, and most rapidly through analytics-centric staff of some type. Critically, though, they’re mostly proceeding with a classically hierarchical (vertical) structure. As a result, specialization is now at an all-time high.

40 years ago, a general manager’s duties extended to such perfunctory tasks as preparing operating budgets and arbitration presentations. In 2016, those may be functions of people who report to people who report to the GM (and perhaps the GMs themselves are reporting to yet another non-owner above them). Even just 10 years ago, well into post-Moneyball times, general managers would occupy barstools in the open lobbies of winter meetings hotels, and talk trades over drinks. That’s simply not how it is any longer, as the mechanics of trades (and evaluations) are more sophisticated.

Certainly, not all organizations have taken the vertical route. But many (or most) have, and it has several effects, as one might imagine.

As organizations seem to be relying more and more upon analysis, the ones actually creating/executing that analysis tend to be relatively low on the chain. Typically the person developing your pitching metrics or developing your interpretations of defensive data — in other words, some of the most critical judgements your organization makes — is beneath the structure which existed prior to the “analytics boom” of the past few years, as seniority plays a large role in hierarchies.

Often, then, the analysts in these departments may have a larger understanding of the concepts and data than the people they’re reporting up to, and it is incumbent upon these analysts to make their conclusions palatable for decision-makers who aren’t as comfortable with the technical aspects of the analysis. As a consequence of this expanding vertical structure, the question of “is this actually valid work or not?” is more difficult to test internally, unless the organization assigns multiple analysts to double-check each other’s work, and doesn’t force one to report to the other.

Upper management may have the impression they’re successfully implementing sabermetric tools — sub-metrics, projections, valuations, etc — due to the presence of analytics staff. They may or may not be, and in situations in which those managing the analytics staff do not have technical backgrounds, some in leadership roles may not have firm reference point by which to judge its relative quality. So while organizations at large may be relying on “analytics” more than ever, the team-to-team disparities in terms of quality can be just as exposed as ever, although the laggards are likely to be more content with their adoption of analytics than previously.

A small number of organizations have managed a very flat approach. There are GMs, AGMs, and Directors of Baseball Operations who have worked neck-deep in analytical development themselves. Not coincidentally, these organizations tend to create environments where analysis is not a department within the house, but analysis of baseball is the house itself. Consequently, virtually all management personnel are not only familiar with the fine details of, for example, the how-and-why of how the organization distributes credit for called strikes between the catcher and pitcher (which of course helps determine the value of players generally), but are expected to question it, share ideas about it, or participate in further development of the matter.

Crucially, though, it’s not that these flat organizations are utilizing less scouting information than the verticals; it’s that they’re treating baseball analysis as the enterprise through which all matters flow, rather than analysis being something that some people do in one department among other departments. The nexus between what are classically termed as scouting and sabermetrics is growing ever tighter. Perhaps, and hopefully, out of the ultimate recognition that they’re the same thing, analysis of baseball and ballplayers. Many scouts in the stands today know what DRS stands for, to say nothing of WAR. The first thing (and perhaps best thing) young analysts are eager to do is pick up on intricacies or wisdom from scouts. A tension still exists, but it’s far less palpable today than it was even a couple years ago.

There’s some question as to how closely teams would want these two departments to cooperate, however. While there’s an obvious benefit to learning for each side, and raising the education level of all participants in general, there’s also likely something to be said for independence and diversity of thought. By creating too monolithic of a union between the two fields, some fear the risk of losing diversity of thought or independence of observations.

Technology is at the epicenter of this nexus. In tomorrow’s follow-up, , we’ll discuss how many of the new tools — Trackman, Statcast, wearable tech devices, etc… — are playing a role.

For teams building departments, the duties of analytics can remain highly conflated and ambiguous. Speaking in generalities, there are three fields within sabermetrics, which have some overlap with one another:

— Data Scientists: Those who manage and manipulate data for the purpose of divining relationships, seeking evidence, and optimizing behavior. Pitch sequencing studies, defensive positioning strategies, and catcher/pitcher stolen base credit allocation are some examples of projects they might work on.

— Developers: Those who have the skills to create/manage systems and structures to display/warehouse/input information. For example, creating internal databases, scouting applications that combine reports with data, and heat maps that coaches can use in their game preparation.

— Predictive analysts: Those who model information and learned intelligence for the sake of judgement/expectations. Player projection systems, player development theory, and player performance metrics would fall under this category.

Perhaps a valid analogy is a Formula 1 race team: there’s a driver, a mechanic, and a designer/engineer. They understand plenty about each other’s jobs, but expecting one to execute the role of the other entirely would be suboptimal at best and perhaps a disaster. (And certainly, in hiring, viewing each of them as interchangeable “car people” would be a very poor idea).

The role of the General Manager also has shifted. In more horizontal organizations, the work of the GM (in an evaluative sense) tends to be a finished product which the GM merely executes. The GM’s value-added, in this case, is the degree to which he’s able to extract value from competing GMs. Negotiation and executive skills are of utmost importance in adding value at that level. In the more vertical organizations, the role of the GM is still far more concerned with evaluation itself, and they can set themselves up as the sole decision maker, with the rest of the staff serving a support role. And, of course, GMs and organizations can act differently depending on the context of the decision.

If we’re talking about a 19-year-old, we gravitate more towards the wisdom of the scouts. If we’re concerned with a 28-year-old with seven pro seasons and five in the major leagues, our answers are perhaps better founded in the performance record. If it’s a 24-year-old whose apparent abilities have never quite matched his performance results, we seek evidence to reconcile where our expectation should lie. In all cases, medical and perhaps other information is considered, and in some cases it’s of great weight.

It’s equally obvious as it is critical: the above is eminently ripe to move from the GM’s head to a more transparent evaluative framework. That’s because player-to-player talent differences are qualitatively small, and the amount of information relevant to their ranking is very large. The essence of sabermetric improvement is moving from fragmentary evaluative processes and decisions to systematic ones. Again, the horizontal organizations are in situations where the General Manager is the executor, rather than the evaluator.

That’s what we’ve seen take place in the world of finance over the past 15+ years, and for the exact same reasons, to the point where financial evaluations are themselves now highly automated, even despite the need for reliance upon different information sources situation-to-situation, often including subjective/unstructured information, which is synthesized within a matter of comprehensive evaluation).

A number of teams have begun to move systematically. The Tampa Bay Rays have been a systematic organization for years, and they’re now gaining some company. Those organizations are reaping enormous benefits, at least in decision-making power if not yet in on-field wins. Organizations which have grown vertically may not gain at nearly the same pace as their competitors, even if they’ve added more staff. The structures built within them will likely need time to calcify before the need for even further change becomes apparent.



Print This Post



Dave is the Managing Editor of FanGraphs.


Sort by:   newest | oldest | most voted
Robert
Member
Robert
4 months 9 days ago

Great read. Thank you for the guest post.

rolliesmustache
Member
rolliesmustache
4 months 9 days ago

Love the work NEIFI does. They’re a must follow on Twitter.

Westside guy
Member
Member
Westside guy
4 months 9 days ago

Interesting stuff – thank you, NEIFI, for sharing it with us.

Richie
Member
Richie
4 months 9 days ago

Well, I never enjoy playing the Bad Dude, but as might be historically obvious, I don’t much mind it, either. I have a Master’s in Economics, have done a small bit of business research and this gave me a headache. I’ll grant I got 80% of the way through the article before I mentally checked out, but dudes! Watch the jargon, the excessive verbosity (e.g., ‘most’ always kicks heinie out of “upmost”) and here and there toss an example in of what you’re talking about.

Richie
Member
Richie
4 months 9 days ago

Yeah I know you did this gratis, but like most all things writing style is habitual. Especially when you’re simply cranking out a freebie. You’ll maximize your professional verticality by cultivating good scribular habits rather than bad ones.

Tulkas
Member
Tulkas
4 months 9 days ago

Super interesting. I love it. It would be helpful to get a few examples, though. I’m guessing a team like the Red Sox is highly vertical, and a team like the Mets or A’s is more horizontal, but I can’t really tell for sure.

waldthm
Member
waldthm
4 months 8 days ago

Describing the data (e.g. measurement) is great and important, but it only provides incremental power in predictive models, which is what’s arguably more interesting.

Here’s something to think about: optimal (player) contract design. What type incentives maximize (future) player performance is not very well understood (to my knowledge). Existing “studies” are fraught with identification issues so that they seem to describe correlations than establish causal relationships. A relevant issue here is how labor market risk for baseball players affects performance, given that they cannot apply their skills to other professions. There are academic studies that show that CEOs don’t put in as much effort or take as much risk (as they should) because of it. I wonder if the same is true with baseball players. If so, can contracts be structured to help them hedge it?

Johnston
Member
Member
Johnston
4 months 8 days ago

Baseball, one of America’s greatest features, has now met bureaucracy, one of its very worst. Expect bad things in the future as a result.

CRPerry13
Member
CRPerry13
4 months 8 days ago

Great post. “Often, then, the analysts in these departments may have a larger understanding of the concepts and data than the people they’re reporting up to, and it is incumbent upon these analysts to make their conclusions palatable for decision-makers who aren’t as comfortable with the technical aspects of the analysis.” So, just like every engineering firm or tech-oriented company I’ve ever worked for.

Joe Blow
Member
Joe Blow
4 months 8 days ago

Most analytics people strike me as the Sales people in your company — they feel that they’re the most important part (because they, you know, ‘make the sale’), but don’t care to even try to see the back-end stuff that goes along with it.

Sales Guy: I just made a $1 million sale!
Accounting: Yeah, but the cost to manufacture those parts was $950,000.
Sales Guy: So I made $50,000!
Accounting: Shipping alone was $60,000. Not to mention all of the people involved with prepping, packaging, billing, collecting, reconciling, filing taxes, etc. You lost us money.
Sales Guy: I just made a $1 million sale!

That’s what analytics guys think they are — they’re not a useful component, they are *THE* component, and as soon as everyone freakin’ agrees with them, baseball will be so much better!

wpDiscuz