Effective Relievers And The Texas Rangers Bullpen

Closer Frank Francisco locks down a save against the White Sox on Saturday, May 2nd.Several weeks ago, an article posted here revealed that the average fastball velocities of major league starting pitchers are tightly correlated with performance. The same study design was used to evaluate relief pitchers and the results were surprisingly different.

FanGraphs was used to capture the Pitch f/x data for all relief pitchers who logged at least 50 innings in 2008. The pitchers were sorted based upon the average velocities of their fastballs. The rolling averages of the ERAs, FIPs (fielding-independent ERA), K/9 (strikeouts per nine innings), BAA (batting average against), HR/9 (home runs per nine innings) and LOB% (left on base percentage) for 10 relievers at a time were calculated and plotted, and are presented in the two graphs below. An explanation of rolling averages is provided at the bottom of the article:

Unlike what was observed for starting pitchers, the various performance metrics of the relievers show very little correlation with fastball velocity outside of the top 15 or so relievers. Most surprising is the lack of an inflection point at average fastball velocities of 91 mph. With starting pitchers, a sharp decrease in performance is observed among pitchers whose fastball velocities average less than 91 mph (see below):

STATISTICAL AVERAGES -- STARTERS VS. RELIEVERS

The discrepancy in starters and relievers led me to seek to better understand the differences between the two classes of major league pitcher. Below are the 2008 statistical averages for the 106 starters who pitched at least 150 innings and 136 relievers who pitched at least 50 innings. For the most part, the numbers bear out the stereotypes -– relievers throw a little harder, strike out a few more hitters, and give up a few more walks.

Numbers that stand out are the significant differences in average wins above replacement (2.6 WAR for starters vs. 0.5 WAR for relievers), batting averages against, and ground-to-fly ball ratios between starters and relievers. In terms of value, it is worth noting that the league's best reliever, Mariano Rivera, had a WAR in 2008 (3.1) that was equal to the league's 42nd-best starter, Kyle Lohse. If you are beginning to warm up to the idea of Neftali Feliz in the bullpen, then those numbers should make it a bit easier for you to wait another year for Feliz to work on his secondary pitches in AAA-ball:

[Editor's note: Two columns were omitted due to space limitations -- BABIP (.300 for starters, .294 for relievers) and LOB% (73 percent for starters, 75 percent for relievers).]

PITCH USAGE -- RELIEVERS VS. STARTERS

A quick comparison of the pitches thrown during 2008 supports another commonly held notion -- relievers feature fewer pitches than starters (see table below). Interestingly, very few relievers simply throw fastballs. Like starters, they throw fastballs approximately 60 percent of the time. The difference is that relievers typically throw one or two additional pitches, whereas starters tend to use two or three additional pitches:

SEPARATING GOOD RELIEVERS FROM GREAT ONES

To get a sense of what performance metrics are important for relievers, I classified the pitchers who tossed at least 50 innings out of the bullpen during 2008 based upon their end-of-year WAR. Statistical averages for the 25 pitchers with WAR above 1.2, 36 pitchers with WAR between 0.7 and 1.2, 31 pitchers with WAR between 0.2 and 0.6, and 44 pitchers with WAR less than 0.2 were calculated (see table below).

To determine which metrics were most significantly different between the 25 pitchers who comprised the top group and the remaining 111 qualifying pitchers, I used the Student's t-test to calculate p-values. The statistical categories displayed are those that had p-values less than 0.05. As noted in the table below, WHIP, HR/9, and BAA topped the list of statistics that separate the most valuable relievers from the other members of the bullpen:

The fact that WHIP was at the top of the list is not surprising given the very strong relationship between that statistic and runs allowed. By extension, BAA, K/9, and BB/9 all contribute directly or indirectly to WHIP and runs allowed. Perhaps the most interesting statistical separator of great relievers is home run rate. Among the top 25 relievers in 2008, only Carlos Marmol gave up as many as one homer per nine innings pitched.

The league's elite relievers limit home runs through a combination of fewer batters per inning, more strikeouts, and fewer home runs per fly ball. As shown in the GB/FB column, it is true that the best relievers have slightly higher ground ball rates than the other relievers, but it is not statistically significant. The reason for this is an apparently strong selection bias for relievers with high ground ball rates. The average GB/FB ratio among relievers is 1.45 and only 37 of the 136 relievers considered in this study had GB/FB ratios of less than 1.00 in 2008.

Although I had expected elite relievers to be especially good at stranding runners, it appears that they are not significantly better than other major league relievers. While it is true that the best relievers strand runners at a slightly higher rate, it appears to be more due to their greater ability to limit base hits than a special ability to pitch with runners on base.

THE RANGERS RELIEVERS (2009)

As Joey has pointed out, the Rangers bullpen has been among the most effective in the major leagues this year. But as you can see in the table below, the bullpen has succeeded without the benefit of more than a couple of pitchers with statistical profiles that are in line with the top relievers from 2008. The rest of the pitchers in the pen look to be getting the job done with a healthy dose of smoke and strategically placed mirrors:

C.J. Wilson has been great and his statistics show why. He's limited homeruns and induced a boatload of ground balls. His strikeout rate and BAA have been slightly above average. Darren O'Day has also been great, though his performance has benefited from a 90 percent strand rate. His ERA won't stay below 2.00, but he looks like a long-term keeper.

Frank Francisco's strikeout and walk rates, as well as his BAA, have been spectacular. It is a good thing, because his ground ball and home run rates have been atrocious and are unlikely to improve much given his career GB/FB (0.65) and HR/9 (0.9).

Jason Jennings and Doug Mathis are well below average in strikeouts, but they are getting the job done by keeping the ball in the park and on the ground. Mathis isn't going to maintain a 0.00 ERA and .127 BAA, but he is profiling as a decent long reliever.

With low strikeout rates, relatively high walk rates, high strand rates, and low ground ball rates, it appears to be just a matter of time before Eddie Guardado and Jason Grilli begin to fail. Despite uninspiring major league results, Dustin Nippert's minor league numbers (0.5 HR/9, 8.7 K/9, 3.3 BB/9) suggest that he might be able to succeed in the bullpen.

FUTURE RANGERS RELIEVERS

Warner Madrigal, Thomas Diamond, and John Bannister are the relievers on the Rangers' 40-man roster who are not on the active roster. Madrigal has done an excellent job this year improving his control (2.1 BB/9 in 2009 vs. 4.0 BB/9 in 2008). Unfortunately, his GB/FB (0.82) and HR/9 (0.9) rates in AAA-ball suggest that he has further improvements to make before he is ready to become an above-average reliever. Diamond's strikeout (9.0 K/9) and home run (0.4 HR/9) rates this year have been good. Unfortunately, his walk and ground ball rates (6.0 BB/9 and 0.58 GB/FB) have been bad. Bannister appears to be a lost cause, as he has put up a 1.2 HR/9, 5.7 K/9, 4.5 BB/9, and .269 BAA in AA-ball and AAA-ball.

Pedro Strop is perhaps the most intriguing reliever in the Rangers minor league system. Some of his minor league numbers are outstanding (11.0 K/9, 0.7 HR/9, .224 BAA, 1.45 GB/FB). Unfortunately, his lack of control [4.5 BB/9 (career) and 5.2 BB/9 (2009)] remains an area that must be improved before he is ready to pitch in the major leagues.

Though I don't particularly like the idea, the minor league statistics of Neftali Feliz suggest that he could be an elite reliever. In his minor league career, Feliz has given up 0.2 HR/9 and 3.9 BB/9 while holding opponents to a .202 batting average and striking out 10.2 batters per nine innings. With the exception of the walks, those are the numbers of an elite reliever.

Kennil Gomez is another pitcher with a minor league profile that suggests that he could be a great reliever. Though most of his stats were produced as a starter, Gomez's career minor league numbers (0.6 HR/9, 8.3 K/9, 3.4 BB/9, 1.90 GB/FB, .245 BAA) would look very good as a major league reliever. In his career, Gomez has been much better against right-handers (1.32 WHIP) than left-handers (1.56 WHIP). A bullpen role would allow Gomez to pitch more to his strength.

Four other starters in the Rangers' minor league system have been discussed as future relievers: Kasey Kiker (0.8 HR/9, 8.9 K/9, 3.9 BB/9, .236 BAA), Guillermo Moscoso (0.6 HR/9, 9.1 K/9, 2.0 BB/9, .226 BAA), Blake Beavan (0.7 HR/9, 5.3 K/9, 1.5 BB/9, .250 BAA), and Tommy Hunter (0.8 HR/9, 6.0 K/9, 2.0 BB/9, .262 BAA). As with any minor league starting pitcher, it is possible, if not probable, that the pitchers' statistical profiles would change with a move to the bullpen and a promotion to the major leagues. Assuming they did not change significantly, I have reservations about bullpen roles for all four pitchers.

Kiker's and Moscoso's fly ball tendencies (0.82 and 0.64 GB/FB in 2009, respectively) suggest that both pitchers might have problems avoiding home runs in the major leagues. The list of relievers who had GB/FB rates below 0.90 and K/9 rates below 9.0 in 2008 is neither very long nor very inspiring: Guardado, Dan Wheeler, Rafael Betancourt, Javier Lopez, Russ Springer, Hideki Okajima, Jon Rauch, Matt Capps, Justin Speier, Doug Waechter, Bob Howry, Joel Peralta, Jesse Carlson, Jesus Colome, Denny Bautista, Taylor Buchholz, Huston Street and Jensen Lewis.

Unlike Kiker and Moscoso, Beavan and Hunter have enjoyed very good ground ball rates in the minors (1.30 and 1.22 GB/FB, respectively). Unfortunately, their low K/9s and high BAAs suggest that they would likely top out as middle relievers. In 2008, relievers who had fewer than 8.0 K/9 and .250-plus BAAs posted an average FIP of 4.43. All of the other relievers posted an average FIP that was almost a run better (3.65).

The 2009 minor league numbers for a few other minor league pitchers with possible big league futures in the Rangers' bullpen:

[Rolling Averages: The rolling averages used to produce the first three graphs used stats from ten pitchers at a time. The first group of ten pitchers comprised those who had the ten highest fastball velocities. The second group comprised the pitchers with the 2nd through 11th highest fastball velocities. The third group comprised the pitchers with the 3rd through 11th highest fastball velocities. And so on through the final group which comprised pitchers with the ten lowest fastball velocities among qualifying relievers in 2008.]

Comments

[Jesse]

I'm coming around more and more on this "Feliz as a reliever" thing... Joey, in your opinion, was the move to the bullpen for him more out of necessity of the big league club this year or because the Rangers also see him as an elite reliever?

[david]

Feliz has a chance to be an elite reliever, but his value would be much greater if he is able to be an effective starter. The real question with Feliz seems to be whether his breaking ball and change-up will develop enough consistency to be effective against major league hitters. Given his youth, I must assume that the Rangers have not given up on him as a starter and are merely using teh bullpen as a way to get him exposure to major league hitters in 2009.

[Robert Bolyard]

Would love to see Murray get another shot with the big club as a left handed option and the occasional spot starter.

[Dave H]

David, could you break down the elite (top 10) closers based upon velocity. Outside of Trevor Hoffman, I don't recall many who throw below 93 or so (much less below 90). I just thought a breakdown of closers would be interesting as well.

As far as Feliz goes, I'd be fine with him eventually being a closer, but I think we owe it to ourselves to let him develop a little longer and show us what he has as a starter.

[david]

Dave - According to FanGraphs, the average fastball velocities for the top 15 relief pitcers ranked by WAR in 2008 were:
Rivera - 93.1
Papelbon - 95.3
Fuentes - 91.6
Lidge - 94.3
Balfour 0 94.6
Broxton - 96.3
Wood - 94.8
Nathan - 93.5
Qualls - 92.6
Thornton - 95.3
K-Rod - 91.9
Ramirez - 92.5
Sampson - 88.9
Francisco - 94.7
Howell - 86.4

Avg fastball velocity for the Top 10 = 94 MPH
Avg fastball velocity for the Top 15 = 93.1 MPH