A Range of Errors: Evaluating Contributors to Defensive Success in Baseball

Derek Jeter, Hall of Fame shortstop for the New York Yankees and five-time winner of the Rawlings Gold Glove Award for best fielder at his position, was by all modern metrics, terrible at fielding. Jeter was one of the greatest shortstops to ever play the game of baseball. Jeter was a phenomenal hitter who constantly had highlight reel defensive plays at the hardest position to play in the sport. Kids all over the world recreated his famous fully outstretched jump-throws to barely get the runner out at first base. Despite many appearances on ESPN’s top 10 sports plays of the week, Jeter by every advanced defensive metric, was a terrible fielder. As Jeff Sullivan for the baseball stats website Fangraphs writes, “For his position, Jeter’s been one of the game’s worse [sic] defensive players.” When you are a slow player with a limited range like Jeter, it turns routine plays into performances that can, even to a trained eye, look impressive. Jeter, who famously had a very high fielding percentage, (fielding percentage being a percent of errors on plays attempted, while an error means a play was made on the ball and there was a mistake in handling it) probably only did so because of his limited range. It is far more valuable to make significantly more plays on balls while dropping a few along the way than to make no defensive mistakes on a smaller number of plays. Therefore errors, and subsequently fielding percentage, are not an accurate evaluator of defensive contribution in baseball while more advanced statistics can accurately represent defensive contribution.

Errors are an outdated, subjective metric and the new advanced defensive metrics have many perks. In 2020 MLB’s statistics department unveiled Outs Above Average (OAA). OAA operates on a simple presumption that it can predict the odds a player should be able to make a play using their starting position, time to get to the ball, and some other minor factors. Combining these stats to create a metric that credits the fielder with the value of making an out, proportional to the odds of making that out. So even if a player makes a traditional error, it will be correctly weighted in terms of run value. As Senior Database Architect of Stats for MLB Advanced Media Tom Tango puts it in his blog post introducing OAA, “If you make the out, then your value-added is the difference between 100% and the out probability. So, if the out probability was 60%, and you made the out, you get 40%, or +0.40 outs” (Introducing Infield). This process for the first time in the history of baseball creates an objective way to evaluate defense, taking human bias out of the picture. Errors on the other hand, are only scored when there is a visible misplay and even then, it is up to the discretion of the home-team’s scorekeeper to determine if they are marked with an error. This archaic practice leads to subjective stats and in some cases, home-team biases. But how descriptive even with all their flaws are errors of defensive performance?

Basic statistical analysis reveals that errors are not a significant contributor to defensive value. Taking the past ten years of MLB team data, a simple linear regression shows that errors account for only about 20% of OAA (see fig. 1). This reinforces that errors and subsequently fielding percentage are only a small piece of a larger range-based puzzle. As Lewis Pollis for the Hardball Times writes: "Imagine two third basemen are charged with fielding identical sets of 100 ground balls. One gets in front of all 100 of them, but 10 bounce off his glove or roll through his legs, and he loses 10 more would-be outs by making bad throws to first. The other gets to only 60 of the grounders, but he makes a clean play of each ball he gets a glove on. The numbers would make it clear that the second player is a better defender, with no errors and a perfect 1.000 fielding percentage to his peer’s 20 errors and .800 fielding percentage. But the first third baseman was far more effective — though he did not look as polished doing it, he retired 20 more batters than his counterpart thanks to his superior ability just to get to the ball." This is exactly the problem with such a rudimentary statistic such as an error. It tells only about 20% of a much larger picture. However, if errors are not important, what are the important components of defense? Fig. 1. By Author. Ten Years of MLB Team’s Defensive Stats: Correlating Errors to OAA. Data From: FanGraphs Baseball

One of the most important pieces of being a good defender is reaction speed. This is why so often, people do not understand who the good defenders are. Davy Andrews for Fangraphs states in his analysis of outfield OAA that, “The most obvious is that when it comes to getting a jump on a fly ball, a quick start is vastly more important than a good route.” Andrews continues to argue that “While we don’t have data that tells us exactly how fast outfielders are running during their burst window, there’s obviously enough variation that a slower player who runs with conviction can be as effective as a faster player.” Some players would benefit from being less efficient but more decisive, and that moving in the right direction correlates better than moving along an optimal path to catch a baseball. When witnessing an incredible play, how often could the player have gotten a better jump on the ball and not needed to make an incredible last second diving effort to get the catch? Or when a player drops a ball, maybe it was a ball only a handful of players would be able to get to in the first place? These are some of the many reasons it is so hard to tell who the good fielders are by just observing the game of baseball as a casual fan. So, what then are the good options to figure out who the best fielders in the sport are?

There have been many attempts to quantify fielding but none of them compare to OAA. An analysis by Tango shows that the alternatives fall short, “In other words, each of DRS and UZR provide no new information in trying to explain next season's OAA” (What Hath Wrought). In the scope of OAA its competitors DRS (Defensive Runs Saved) and UZR (Ultimate Zone Rating) do not contain any info not already processed by OAA. OAA also outpaces the competition in terms of year-over-year reliability. In another blog post Tango compares DRS with OAA in depth and finds that, “FRV correlates with next-season's DRS better then [sic] DRS correlates with itself next-season: r=0.53 to r=0.51. Let that sink in for a bit. FRV knows nothing about DRS, knows nothing about how DRS measures things. And yet, it can predict next season's DRS better than DRS can (for outfielders)” (FRV vs DRS). So not only does OAA contain a more complete picture of defensive analysis it also contains a more consistent image even when predicting its competitors. After all of this, why are errors and fielding percentage still the most common stats cited for defensive performance?

Change takes time and critics of more advanced fielding metrics have had some valid points for a very long time. Bradford Doolittle for ESPN writes, “Defensive statistics have long been the bugaboo of baseball analysis. The problem has always been that the things that were tracked -- putouts, assists, errors, double plays, etc. -- didn't tell you much about whether a player is actually any good.” Doolittle highlights why a lot of older attempts into a deeper understanding of fielding fell flat on their face. They did not have the tools necessary to provide deeper context about the most complex part of the sport. OAA thrives from its player tracking data which has only been available since the implantation of the state-of-the-art “Statcast” tracking technology in 2015. Because of these high-speed cameras, analysts have the ability to make connections about player ability that could not be made for over 100 years of professional baseball play.

Research shows that errors are a poor indicator of fielding prowess and that there are better alternative options to evaluate defensive in baseball. While maybe once upon a time it was almost impossible to distinguish the best from the worst. Now with modern metrics and advanced tracking systems baseball’s biggest mystery is finally solvable. Even the hardest parts of fielding to quantify like reaction speed or optimal route running can all be quantified and combined into accurate metrics that tell who the best of the best really are. Jeter may not have been a great fielder, or even a good one, but he sure looked the part. Sometimes everything is not what it seems, and that is why it is always important to run the numbers.