My friend alerted me a recent article by Noah Davis and Michael Lopez on improvements in umpiring published at fivethirtyeight.com. I’ll comment on the article more in a moment, but reading it gave reminded me that I wanted to write about a related topic: the historical event last month in which a computerized camera system called balls and strikes in a professional baseball game. The two tie together, as you will see.
As reported by CSN Bay Area, an independent league team called the San Rafael Pacifics played two games using the Pitch F/X system to call balls and strikes instead of a human umpire. Even the casual fan know Pitch F/X, even if they don’t recognize the name. That little square in the bottom corner of the TV broadcast depicting the placement of hte pitch in relation to the strike zone? That’s Pitch F/X. If you’ve played around with the the gameday functions within the inimitable MLB At-Bat app, you might have come across data on velocity and trajectory of individual pitches. That’s also Pitch F/X. As the above-referenced article notes, MLB installed Pitch F/X into every stadium in 2008. I think of Pitch F/X as a bunch of high speed cameras and some software that does physics calculations.
I don’t know why, but when I first heard about these Pitch F/X ump games, I immediately assumed that these would be the first professional games ever without a home plate ump. But I was wrong on that point, as there was still a home plate ump present, because obviously, there are still calls to be made besides pitch placement, such as plays at the plate, check swings, batter time outs, etc. More importantly, however, I knew they would be the first games to run the incredible experiment of having a completely accurate, impartial, and unwavering strike zone for every pitcher and batter. Wonderful baseball podcaster Paul Sullivan has discussed this fantasy on at least one occasion. I’m paraphrasing him when I say that, as instant replay has made the game better, the prospect of eliminating the subjective, biased strike zone would make it better still. And the technology exists to do it.
PitchF/X was employed Tuesday and Wednesday, July 28th and 29th, 2015 in two games between the San Rafael Pacifics and the Vallejo Admirals. The games were fundraising events spearheaded by former MLB OF (and very cool guy) Eric Byrnes. (I tend to associate Byrnes with his years crashing into outfield walls with the Oakland A’s and to his post-retirement stint playing softball for the Dutch Goose in Menlo Park, CA. I was at Stanford at the time, so I knew the bar and thought this was incredibly cool. Now, of course, copy cats like Yankee great Hideki Matsui play softball too.) The goal was to raise money for the Pat Tillman Foundation and, as I learned during the game, Byrnes donated hundreds of dollars for each strike out, which added up to a sizable donation. Kudos to him. But I was there for the Pitch F/X experiment as were many other baseball nerds and journalists in the crowd, easily identifiable among the crowd with laptops open wide.
It was my first time attending a Pacific Association game, so first a few thoughts about overall experience. It was great. The drive was a reasonable 20-25 min from San Francisco, though one will likely hit evening commuter traffic on a week night, as I did. The ball park, Albert Park, sits snugly within a larger community park, and it reminded me of a few Single-A parks I’ve been to. It probably sits around 2,500 fans and there is premium on-field seating that allows fans to essentially share space with the players. There are no proper dugouts; the players sat on plastic lawn shares in the vicinity of the on-deck circles like a classroom of restless pupils. There are no bad seats in the house and I loved the small town, family atmosphere of the game, i.e., the local business advertisements, young folks working the refreshment stands, the local girl who sang the national anthem, and the players being within arm’s reach of fans and kids. I bought a general admission ticket and settled in a few rows up from behind home plate. Personally, I didn’t love the protective netting that spans the entire grand stand because it’s visually disturbing for me to constantly look through it. That said, I do recognize its value for fan safety.
The quality of play was pleasingly good. I didn’t look them all up, but clearly the players were guys who played college ball or were drafted and played in the minor leagues. I saw hard hit balls, a home run, and some acrobatic plays in the field. A couple notable names for you aficionados: Admiral’s 1B was Lydell Moseby, son of the great OF Lloyd Moseby. The Admiral’s DH was PJ Phillips, brother to current Reds 2B, Brandon Phillips. Phillips was one of the older players, but he made solid contact and anchored the Admirals order.
OK, what was like to have PitchF/X call balls and strikes? It was surprisingly unobtrusive. It didn’t disrupt the pace or even the feel of the game. Here’s how it worked: Before each pitch, the umpire crouched behind the catcher as usual. Byrnes sat in the stands with a monitor in front of him with a simulcast showing the pitch placement, just like one sees watching a television broadcast. As the pitch arrives, Byrnes would simply speak the call into a microphone for all to hear – ball or strike. There was the ump, but Byrnes was called the “arbiter”. It was simple, even as implemented in such as technologically-deprived setting as Albert Park. Imagine what a Major League park could do with their Jumbotrons and secondary scoreboards placed around the stadium.
I went to the game thinking what a neat experiment I would see, never believing in a million years that MLB would ever adopt such a system. Now, having seen it in action in an understandably crude form, I feel differently. I could easily see a Pitch F/X system working in a MLB game. After an inning or two, I more or less forgot it was there.
Here’s an video example of how the system worked:
Early in the game, the home plate ump would instinctively make a pitch call. It was fantastic because, playing to the crowd, Byrnes would say over the PA, “Do you think you were right?……Barely!” A whoop from the crowd. Other times, when a batter struck out looking, or took a strike, Byrnes would make the call and tell the batter exactly where the pitch was. He also did this to the pitcher when he didn’t get a call; “Sorry, man, that was about three inches outside.” It was fantastic. Not only would this system essentially make strike zone dispute go extinct, it could also be used as an incredible teaching tool. Batters and pitchers could sharpen there perception of the strike zone, either in real-time during game as in summary form after specific games or many games.
Want proof? Because Pitch F/X data have made human umpires better. That’s the main takeaway from the article by Davis and Lopez I mentioned at the start of this post. Since 2008, with feedback on performance using Pitch F/X data, umpires have shown steady improvement in the ability to accurately call balls and strikes. Well, actually, the data show accuracy improvements on strike call, while ball calls have stayed fairly steady. Importantly, while there is considerable variability across umpires in their performance accuracy, all umps appear to show an improvement. That is, the best have gotten better and the worst have better too (at calling strikes accurately).
BUT….. even so, the very best umpire, Lance Barksdale, calls pitches accurately at just under 90% of the time. As Noah and Lopez point out,
“The difference between Barksdale and a league average ump is about five correct calls per game; the difference between Barksdale and the league’s worst umpire is closer to 10 calls a game. On average, that’s about one judgment call per inning that a good ump is getting right and a bad ump is getting wrong. That might not sound like much, but if once every six outs a batter gets another swing after a third strike that wasn’t or a pitcher strikes a hitter out on a pitch that’s actually a ball, you can start to see the impact.”
What Davis and Lopez are getting at is the benefit of having the best ump versus a league average ump (5 call improvement) or versus the league worst ump (10 call improvement). But how big would the improvement be if we compared the best ump, Barksdale, with Pitch F/X? Well, we’d capture that extra 10% that he is still missing, which if my numbers are current, is about 15 pitches across a 9 inning game. (The average number of pitches thrown per game, which according to this post sits around 300 per game (150 x 2). I haven’t been able to find the breakdown between “called pitches” vs “swinging pitches”, i.e., either put in play, fouled off, or swung at and missed. But lets say that 50% of pitches require the home plate ump to make a ball-strike determination. That would mean 150 pitches across both teams. Even with Barksdale behind the plate, 10% of pitches, or 15 pitches, would be called inaccurately.) I agree with Davis and Lopez’s argument that the performance gap of 5-10 pitches per game between umps strongly suggests that the best umps, e.g., most accurate, should be given preferred assignments in big games, like playoffs. But if you believe that, then I hope you will be compelled by the possibility of an even larger improvement in pitch calling accuracy by adopting a Pitch F/X system.
But what about the umpire’s union? Surely they’d never go for this, right? Well, I actually spoke with the home plate ump after the Pacifics game to get his impression on what it was like to be relieved of his pitch calling duties. His answers? A little odd due to force of habit, but it was fun and he could get used to it. There is still lots to do as umpire to earn your paycheck. Maybe this really could happen, I thought to myself………
One other note about the game:
Around the 4th inning, something occurred to me: there were no defensive shifts. It didn’t matter who was batting, the defensive positioning was straight away. How rare in today’s game?! If you follow MLB as closely as I do, you know that defensive shifts have become a regular, accepted part of the game (which is fine with me – it’s evolution baby!) To see teams not shifting now seems odd. But for the four teams of the Pacific Association, how could it be any other way? Sure, they see each other a lot, but they must rely on “traditional” scouting reports, perhaps limited data, and definitely in-game experience. Not enough to support defensive shifts.
Update on this old post:
MIT Technology Review recently published a brief article entitled, Baseball players want robots to be their umps. The article notes that Chicago Cubs super-utilityman Ben Zobrist is a “vocal supporter” of automated balls and strikes calls.
“That’s why we want an electronic strike zone.”
—Zobrist, shortly before getting his first career ejection
This is the first time I’ve heard of an MLB player explicitly stating a desire to move away from human umpires on this issues. Now three years since I wrote my initial post, I’d say that it’s inevitable that human umpires will not call balls and strikes in the not-so-distant future. Instant replay in baseball is nearly 10 years old now, having been expanded to cover more of the game’s trickier situations (close calls at bases, fair/foul, etc). Frankly, it’s made the game better. Self-driving cars are the natural culmination of the progression of automation in car driving that’s been going on for decades. I predict that “robot umps” will be the culmination of technological progression in baseball, with the fringe edge cases still requiring human review.