Play Ball! LA Dodgers sign hyper-converged infrastructure to build a winning off-field game

This article is part of the Technology Insight series, made possible with funding from Intel.

As the 2020 Major League Baseball season draws to a close with the Los Angeles Dodgers and Tampa Bay Rays squaring off in the World Series, nothing is business as usual. The COVID-19 pandemic has left players in testing-constrained bubbles and teams with cardboard cutouts of fans in the stands. For some, this might be a time to plod along and hope that 2021 will be better. But the Dodgers aren’t leaving their future up to hope. They’re turning to a cutting-edge technology overhaul and revamping every facet of their operations with analytics and new, data center-driven experiences for fans and the team alike.

Sports fans know the team’s exciting path to the Championship series. Here’s the off-field story of how the club will fuel new fan engagement and better modeling, so that when the stands fill once again in 2021, the Dodgers will be strongly positioned to deliver better fan experiences and more wins than ever.

Throwing out a slow runner

By 2019, the Dodgers understood that the data center infrastructure driving their analytics could mean, from a franchise profitability perspective, the difference between a game-winning homer and fouling out. And in a swiftly changing sport challenged by patrons demanding new and better experiences, their legacy analytics and operational systems simply weren’t good enough.

Management didn’t just want faster systems and bigger screens at Dodger Stadium; they wanted to evolve how baseball was enjoyed. Contactless technologies would be part of it, including eliminating turnstiles and waiting in lines. Once fans are in their seats, they can enjoy individualized stat feeds and video clips on their phones.

According to Ralph Esquibel, Dodger’s vice president, information technology, the IT department had 38 server nodes comprised of different machines with different firmware running different hypervisors, all trying to connect to the same storage area network (SAN). Business units were complaining about slow job execution, with some models taking a day to run. The infrastructure filled three racks, and so much required maintenance and administration that it stifled the team’s plans and growth. While the Dodgers are a big-name franchise, they’re also a small business. The team employs 550 workers, including an IT staff of fourteen and just two systems administrators.

Esquibel called in Cisco, which pitched its HyperFlex hyperconverged infrastructure (HCI) platform with Intel Xeon Scalable processors and Intel Optane storage. HCI is an approach to data center architecture. It is designed to simplify management by “de-siloing” servers, storage and other key elements into shared pools of resources that can agilely deliver software services. Cisco maintained that a six-node HyperFlex platform with Optane could outperform the Dodgers’ existing 38-node solution by a wide margin and dramatically alter the time spent on administration. Esquibel went to top management and ultimately got them to agree on the overhaul based on three priorities:

1. Faster data modeling. This would not only improve analytics for fans but also feed directly into player training, lineup, and other facets of play performance, thus improving team results.
2. Maintaining and/or improving uptime. Any downtime on game day could cost many thousands of dollars across ticketing, concessions, and merchandise sales. The more complex and fragmented the data center solution, the more likely downtime became.
3. Streamlining data center maintenance. From component swapping to platform patching, the solution needed to make life easier on IT through greater simplicity and process automation. In short, fewer hours spent fixing and manually updating meant more hours spent on activities that could directly improve the team and organizational profitability.

Early stats are promising

While the Dodgers haven’t yet released before/after benchmarking results, some clues might be found in testing done by ESG in 2019 on Cisco’s HyperFlex platform with and without Intel Optane DC SSDs.

The graph above, from Intel’s brief “Storage at the Center: Giving Hyperconvergence a Boost”, shows synthetic mixed workload performance under Vdbench. The Optane SSD configuration yielded roughly 60% improvement on IOPS and over 35% lower latency. A fuller analysis can be found on Cisco’s site, but the take-away is similar. For OLTP workloads, Cisco found a 71%  IOPS improvement and 37% drop in latency with Optane SSDs. Of course, there were more variable in the Dodgers’ old and proposed solutions, but the general results boded well for big net improvements and achieving their data modeling goals.

Beyond the speeds and feeds of specific components, just updating the data center from a traditional three-tier design to hyperconverged infrastructure promised advantages. An IDC business value study of HyperFlex found 452% five-year ROI and 50% lower cost of operations over five years.

So the team took the plunge. On November 1, 2019, Dodger Stadium started its $110-million renovation. Soon thereafter, with only a 24-hour window for moving the entire production environment, the team had its six-node HyperFlex solution up and running. What had previous consumed three full racks now fit into less than 24U of one rack.

13x faster analytics processing  

The results surprised everyone. Esquibel notes that one of his IT staff approached him complaining that the system had to be broken, because a job that usually took seven hours was completed in only 30 minutes. Overall, the Dodgers found that the HyperFlex solution accelerated analytics processing by 13x. Thanks to the reduction of nodes and CPU cores, it lowered licensing costs by more than 70% percent. Platform features such as HyperFlex’s deduplication, aided by the accelerated storage performance of Optane SSDs, proved instrumental in saving hours of job time.

“Instead of running one job every day, we’re running that job based on new variables we’re receiving on-the-fly multiple times a day,” says Esquibel. “We’re changing our behavior in business.”

This leap in IT infrastructure allows the Dodgers to stop being preoccupied with supporting legacy bottlenecks and instead look forward to tackling new challenges. Little did anyone know at the end of 2019 that a global pandemic would shutter Dodger Stadium and transition the team to working from home on March 8. The MLB regular season, scheduled to start on March 26, was bumped to July 23. With a capacity of 56,000, the only fans in Dodger Stadium stands, including some dogs and cats, were made of cardboard.

Building for a strong 2021

While the 2020 season was disappointing in some ways, it did allow the Dodgers to field test its new HyperFlex solution and treat quarantine almost like off season. No one in IT sat on their hands, waiting idly. Instead, the crew began to prepare for the next season. Whether in days or months, eventually real fans would be back in those seats, and the team can spend its time crafting the experiences they want those fans to enjoy.

“Sports has really changed,” Esquibel says. “The amount of knowledge a fan has when it comes to statistical information, the sexy launch angles and spin rates, everything is really gearing up toward probability and giving individuals as much information as possible to start thinking about gaming inside of the game. All that compute has to take place on-prem, because there can be no latency whatsoever.”

Ultimately, 2020 became the year the Dodgers’ IT focused on systems management and putting new workflows and applications in place. The team knows that, even if it couldn’t bring its updated data center fully into action, successful organizations are always planning for the future. There will be a contactless customer experience ready for a post-COVID-19 clientele newly accustomed to social distancing. Fans will watch player stats update on signage in real time with every new play. The new infrastructure will have the scalability to support multiple devices on every customer and keep every interaction secure. On the player side, faster modeling will make the team tracking and planning more accurate and better able to make changes in real time that can convert into more wins.

Perhaps most of all, the solution will be more flexible, allowing the Dodgers to pivot from yesterday’s slow, limited kiosk experiences into agile, video-driven interactions—and still be ready to pivot again as future needs require. It’s a speedy, flexible tech foundation worthy of a championship franchise.