Analyzing pitchers is one of the most difficult things to do in baseball (at least, in the “non-playing” category). Pitchers are notoriously fickle, and their performances can vary widely from start to start and year to year. They don’t follow a set aging curve like position players (who peak at ages 27-30), but improve and decline with no overarching pattern. Some pitchers are late-bloomers and don’t peak until their 30s (e.g. Randy Johnson), while others peak in their early 20s and never reach the same level again (e.g. Scott Kazmir).
Not to mention, when you try analyzing a pitcher’s results, there are so many variables in play. How much of a pitcher’s performance is his talent shining through, and how much is the defense, opposing team, umpire, catcher, and ballpark? With no discernible difference in his pitch movement, sequencing, or velocity, a pitcher may let up 8 runs in four innings during one start yet turn around and throw an 8 inning shutout his next time out. How much of that variance should we pin on the pitcher and how much is outside his control?
These are all difficult questions without any exact answer, which is why there are a large number of pitching statistics available here at FanGraphs. In order to see past those confounding variables and get a grasp on a pitcher’s true talent level, it’s best to look at a wide range of statistics instead of relying upon one as the be-all-end-all. ERA, FIP, tERA, xFIP, BABIP, LOB%, HR/FB – all these stats tell you something different and paint a more complete picture when used together.