Are you sure you want to spend big money on the bullpen?

The bullpen was a major strength of the 2020 Twins. The reliever group performed at one of the highest levels in baseball — tying with Tampa for the major league lead in fWAR (3.6), finishing fifth in FIP (3.85), strikeout rate (27.4%), and win probability added (2.26); and sixth in ERA (3.62). Fast forward to the middle of this frigid offseason, though, and the bullpen is now one of the Twins’ position groups most in need of strengthening before the 2021 season begins.

Sergio Romo’s $5M contract option for 2021 was bought out at the start of the offseason and he remains a free agent. Tyler Clippard also remains available in free agency. Trevor May became a free agent for the first time and did not hesitate to cash in on a multi-year deal with the New York Mets. Surprising waiver claim success Matt Wisler was non-tendered in early December, but has since latched on with the San Francisco Giants.

Those four were key players in last year’s bullpen. Collectively they worked just more than 36% of the innings covered by Twins’ relievers. They were effective, too, pitching to a 3.11 ERA and striking out just more than 31% of the batters they faced. If the four of them all end up somewhere else for the 2021 season they will be leaving a significant hole in the Twins’ relief corps.

Naturally, then, we are expecting the Twins brain trust to look to the free agent market to acquire good relief pitchers to replace them. The market this winter offered a number of bigger name and proven relievers that have been speculated to be fits for the Twins — like Liam Hendriks, Brad Hand, Archie Bradley and the aforementioned May. But those names have started to come off the board — Liam Hendriks to the White Sox, Archie Bradley to Philadelphia, and May to New York — and the chorus is growing louder and louder by the day for the Minnesota Front Office to “do something.”

Of course, it’s not as though they haven’t done anything to address the bullpen thus far. It’s just that the acquisitions of Derek Law, Glenn Sparkman, Luke Farrell, Juan Minaya, Ian Gibaut, Brandon Waddell, and Hansel Robles are not the kind of big, splashy moves that would count as “doing something” for most of us.

The bullpen has holes, so naturally it makes sense to want to spend money to patch it. Bringing on someone like Hand and his Minnesota roots would make for a fun story and natural connection. Aggressively targeting former Twin Hendriks (who was arguably the best available reliever this winter) also would have made sense. Both even come with the Proven Closer™ distinction that many value highly.

But I’m here to tell you — we don’t want the Twins to spend big money on the bullpen and the idea of a Proven Closer™ is mostly nonsense.

The data shows the track record of big free agent reliever contracts is very spotty and spending significant dollars over multiple years is no guarantee of greater relief production. In fact, money spent isn’t even a useful indicator of relief production.

Recent Free Agent Reliever Spending & Results

I went to the data to see how recent big money and multi-year free agent reliever signings have worked out on the field. In particular, I wanted to see if there has been a positive relationship between spending and production from relievers.

To investigate this question, I pulled together a dataset that included every free agent reliever that signed a major league contract in the last three offseasons (2018-2020). The details of the contracts they signed were sourced from Spotrac’s database and performance data for each player was obtained from Fangraphs.

In total there were 102 free agent contracts included. 63 of them were one year deals, 27 were for two years, 11 were for three years, and just one was for four years. 88 of the deals were for 1-million dollars or more in annual average value (AAV), with the remainder being for amounts closer to the major league minimum or just slightly above it. The largest dollar deal in the set was Wade Davis’ signing with Colorado for 3 years and just more than $17-million AAV before the 2018 season. Left-hander Drew Pomeranz had the longest contract in terms of years, with that four year deal with the Padres before last season.

Of the 102 deals, 90 of them have been completed. Because 12 still have seasons left to play, I decided to use single season data points, instead of cumulative values over the life of the contracts. That gave me a data set with 138 completed reliever seasons to analyze.

One note to point out is the dataset also includes the shortened 2020 season. While that could introduce some noise to the data, most of the key takeaways I’m going to try to make will focus on benchmark thresholds, like above or below replacement or average, instead of cumulative totals that might be skewed by the imbalance in games played. Wins above replacement from Fangraphs (fWAR) and win probability added (WPA), which shows how much a pitcher’s results added or detracted to a team’s chance of winning, are the metrics I’ll use.

Free Agent Reliever Results

What I, and others like Fangraphs’ Craig Edwards, who have done this sort of analysis before found is that spending on relievers is essentially a crapshoot.

Of the 138 completed reliever seasons in the dataset, 51 of them (37%) were below replacement level by fWAR. 58 of the 138 (42%) produced negative win probability added (WPA). As you can see in the histograms below, the results are somewhat normally distributed around 0 in both charts.

The takeaway from these simple plots is that, at least for the past three seasons, relievers signed to free agent contracts have hurt the signing team on the field nearly as often as they have helped it.

That data includes all the signed relievers, regardless of the amount spent on them. Maybe the results are skewed above because of the low-dollar signings? Perhaps factoring in the annual cost of the signing shows a different pattern?

Below, I plotted the AAV of the contracts against the single season fWAR and WPA produced. As you can see, the data is scattershot, with no clear pattern:

In theory, you would expect the data in these charts to show an “up and to the right” trend, which would indicate that higher paid relievers provide better production. After all, the higher paid free agent relievers are supposed to be the best relievers. But that trend definitely isn’t present in this data. Overall, the R-squared values of the two charts, which indicate how well one data set predicts the other (e.g., AAV predicting WPA), are almost non-existent (.005 and .027, respectively).

The takeaways are that there is basically no relationship between spending and performance from free agent relievers and spending definitely doesn’t portend greater production.

Large and Multi-Year Contracts

Those takeaways are surprising. More highly paid free agent relievers are supposed to be better. Just to be sure, let’s focus specifically on the bigger and multi-year contracts. Using the admittedly arbitrary threshold of $8-million AAV, I was left with 44 reliever seasons in the dataset. This includes the Twins’ signing of Addison Reed before the 2018 season. Of these 44, just 6 were single year deals. The remainder were for multiple years.

I looked at this data in the same way as above:

Nothing much changed when the higher paid group is isolated. These are the (in theory) “better” relievers, including many who were among the most sought after on the market when they were available, yet the histogram distributions are again somewhat normal and the x-y plots are still scattershot with R-squared numbers that are comparably small to before.

In this set, it’s even more of a coin flip as to whether their production after they signed their big deals was beneficial on the field. Of the 44 seasons in this group, 18 were below replacement level and 23 delivered negative win probability added. The median fWAR of the group was just 0.05 and the average was 0.1 — essentially replacement level. Even worse, the median WPA was —0.12 and the average was —0.19, meaning these relievers cost their teams chances to win more often than they helped.

Perhaps even more telling of the volatility of relievers is that only 10 of 39 relievers who signed multi-year deals in the past three offseasons were above replacement level in each season of their contract. Four of the 39 found themselves unable to finish their deals due to injury or release (including Reed in Minnesota).

Making Sense

High value, multi-year deals or not, it just isn’t the norm for relievers to consistently perform at the highest levels season after season. To illustrate, here are the lists of top 10 relievers by WPA from the 2018 and 2019 seasons, side by side:

Only four names appear on both, highlighted in red. To extend it further, only two pitchers from the 2018 list also appeared on the 2017 list (not shown) and only Vazquez made the list in 2017, 2018, and 2019. None of the pitchers in the two seasons listed above appeared on the 2020 list (setting aside small sample size variation due to the shortened season).

Part of the reason this plays out like this is because of how relievers are used today. The length of relief appearances has decreased steadily over the past several decades as the frequency of appearances has increased. Teams routinely go to relievers for one inning (or less) stints, which makes a reliever season somewhat of a small sample by definition. A few bad outings over the course of a season can affect a reliever’s topline season total stats.

A byproduct of the short outings is that relievers are incentivized to max out their effort every time they take the ball. While difficult to prove, it seems likely that operating at these extreme levels contributes to pitcher injury or ineffectiveness in time. While injuries certainly will sink a large free agent deal on a relief pitcher, slight declines in effectiveness might be an even larger risk factor for a signing club.

Slight declines in effectiveness disproportionately destroy reliever value primarily because relievers are simply not as talented or good as starting pitchers (in general). Almost no pitchers are developed exclusively as relief pitchers as they are progressing through high school or college. The best pitchers start games at the youth levels and chances are pretty good that a pitcher who is talented enough to pitch in professional baseball was one of the best arms on their team as a youth player. Almost every reliever pitching in the bigs today was a starting pitcher at lower levels.

Most relievers become relievers only after they get into professional ball because, for one reason or another, they were not good enough to be a successful starting pitcher in the Majors. Common reasons for this include lack of command, or not having a 3rd or 4th pitch, or the inability to hold their stuff and velocity for 100+ pitches in an outing.

Whatever the reason, the bullpen offers opportunities for these “lesser” pitchers to thrive, by consistently facing more advantageous platoon matchups, or simplifying an approach to focus on their best pitches, or because command just isn’t quite as important when you can just blow your best max effort fastballs by hitters for a single inning at a time.

The point is, a lot of relievers are already at the max of their abilities by the time they reach a Major League bullpen and then must try to be successful living on the razor’s edge in the highest leverage situations the game has to offer.

All of this makes it very difficult to have lasting staying power.

Diversifying with a Lottery Ticket Approach

None of this is to suggest that spending on relievers in free agency can’t work out. It can and does in individual cases. But in general, it’s a gamble where the risk to reward ratio is hard to empirically justify. Three seasons of data is hardly a complete sample, but it seems that signing a reliever to a big money or multi-year contract has about a 50/50 shot of hurting the team’s chances of winning in any given season and a 75% chance of going bad before the contract is up.

So, I ask: “are you sure you want to spend big money on the bullpen?”

To me, all of this seems pretty clear that the bullpen is a prime place to take an approach that diversifies risk. Instead of tying up a lot of resources for multiple years into one bullpen arm, it seems quite prudent to try to grab several lottery tickets on one-year deals, which is an approach that has worked for the Twins before and that it seems they are trying again.

The fact remains, as the data above bore out, you can get the same production from a much less costly class of players. Actually, you might even be able to get better production. I also filtered the data set just for the contracts and player seasons with an AAV between $1-million and $5-million. Among these, the median fWAR and WPA were both 0.1. Those are not dramatically better than the high-dollar set above, but they are better. When we consider the average fWAR and WPA produced by this class of free agent — 0.26 fWAR and 0.36 WPA — the numbers slant even more in favor of smaller-dollar free agent relievers.

To be clear, I’m not arguing that the Twins front office shouldn’t spend money in free agency. They should (and will, I think?), especially in light of the offseason moves made by the rival White Sox. What I am arguing, though, is that that money they do spend shouldn’t be spent heavily on relievers. Let the White Sox spend $54-million on Hendriks, and the Mets spend $15.5-million on Trevor May, and someone else spend big on Brad Hand. They are good relievers, no doubt. But we have no idea if they’ll be good relievers in the future.

For my taste, I’d rather patch those bullpen holes by bringing back Tyler Clippard, taking that flier on Hansel Robles, and adding another lottery ticket or two like him on one year deals for a few million dollars. Then I’d direct everything else that is available to spend this offseason to the rotation and Nelson Cruz.

---

John is a contributor to Twinkie Town with an emphasis on analytics. He is a lifelong Twins fan and former college pitcher. You can follow him on Twitter @JohnFoley_21.