Beyond the box score: AWS and NFL AI-powered Tackle Analysis

Picture one of the most anticipated matchups of the NFL football season with the clock winding down. The quarterback hands off the football to the running back, who is met by a wall of defenders at the line of scrimmage. In this moment, it’s unclear if the running back will evade a tackle and make a run for the end zone, or instead, be tackled for a loss. Tackling is a huge component of football, but assessing the quality of NFL tackling plays has been largely subjective—until now.

As the 2024 National Football League (NFL) season kicks off this month, fans can dig into new tackle-related metrics calculated with the benefits of artificial intelligence (AI). Developed by the Next Gen Stats (NGS) team and Amazon Web Services (AWS), Tackle Probability paints a more complete picture of the art of the tackle than previously possible, from both an offensive and defensive perspective.

Until now, tackling statistics were limited to simple counts of solo and assisted tackles. A new tackle probability model aims to capture the entire process of tackling, from pursuit angles to missed attempts, providing a wealth of deeper and more granular insights for teams, broadcasters, and fans alike.

Earlier this year, the NGS team enlisted AWS Professional Services to build a machine learning (ML) model that uses more than two million individual data points to compute tackle probability for every player, during each play, every 10th of a second. And the model was trained and in production in less than six months.

“Football is games within games, and it’s been tough to find stats that truly quantify tackling beyond just the outcome. It was a huge hole in our football stats toolbox and an area where we knew we needed more quantified metrics,” explained Mike Band, Senior Manager of Research and Analytics, NFL Next Gen Stats. “AWS gave us the expertise and support to get this new, highly complex stat past the finish line in time for the season.”

Next Gen Stats are just one aspect of AWS’s ever-expanding partnership with the NFL. This season, the League will also see an increase in operational efficiency as it implements Amazon Q Business, a generative AI-powered assistant, to boost employee productivity and enable staff to concentrate on producing high-quality content to engage fans.

Tackle Probability in play

Assume a player has racked up 150 tackles in a season. At face value, that’s an impressive stat. But with data-driven insights, a lot more information is available to unpack and truly measure a player’s performance. How many tackles was the player expected to make over the course of the season? What was the angle of approach on tackles? How close was the player to the ball carrier when they initiated the tackle? Similarly, if a defensive player sets the edge and forces the running back to take an alternate route, they might not make a tackle, but it’s still a great play that can have a meaningful impact on the game.

Tackle Probability helps provide tangible metrics that incorporate these variables and indicate how well a player performs in different scenarios on both offense and defense. Deeper visibility into tackling performance had been an identifiable gap in the ever-expanding NGS playbook, with data that would be near-impossible to measure and derive insights from without AI.

AWS and the NGS team trained the Tackle Probability model on NFL data from 2018 through 2022, and used 2023 season data as the test set. Now, for a single play, the Tackle Probability model processes 20 features for every frame for all 11 players on defense. A ten-second play can easily generate 20,000 data points.

“We fully tested 15 different machine learning models on over a million data points in four months, and now we can run the model in a live setting where it’s going through hundreds of thousands of data points every second, millions per game. That’s impressive,” explained Keegan Abdoo, Research Analytics Manager, NFL Next Gen Stats. “We can compute all those metrics, feed them into a model, have the model compute a time series probability, output that, and create derivative metrics from that all in seconds.”

Together, AWS and NGS have developed more than 50 ML-based stats. These stats can be stacked and layered to gain new, deeper insights. For example, by fusing tackle probability and expected yards models, the NGS team can determine approximately how many yards a defender saves when they make a tackle, and, conversely, how many yards they give up when missing it.

Deriving deeper insights

Along with providing a tackling measurement, Tackle Probability can be used to calculate and apply many other metrics to both offensive and defensive analysis.

Current derivatives include:

• Missed Tackles by a defender
• Tackle Efficiency (percentage of tackle attempts that result in a successful tackle)
• Missed Tackles Forced by a ball carrier
• Group Tackles with three or more defenders contributing to bringing down the ball carrier
• Number of Players in a Group Tackle (indicates how well defenses “swarm” to the ball carrier)
• Open Field Tackle Attempts (no other defenders in a certain radius of the ball carrier)
• Tackled Out of Bounds (ball carrier is forced out of bounds)
• Primary Tackler (initiating player in group tackle scenarios)
• Downhill Tackle Attempts where the defender approaches the ball carrier from down the field
• Upfield Tackle Attempts where the defender approaches the ball carrier from behind the ball carrier’s forward progress
• Chase Down Tackle Attempts where the defender travels a certain number of yards upfield and behind the ball carrier
• Missed Tackle Yards Lost by a ball carrier after a missed tackle attempt to the next tackle attempt
• Tackle Yards Saved after catch or rushing that the ball carrier loses under expectation
• Yards Gained in Group Tackle by a ball carrier from the start of a group tackle until the ball carrier is brought down

Building on NFL’s Big Data Bowl

Since 2018, the annual NFL Big Data Bowl competition has helped shape the sport, informing the creation of new NGS centered around running backs, defensive backs, special teams, and pass rush plays; this year’s hackathon style event planted the seed for Tackle Probability. Data scientists of varying levels of expertise converge at the Big Data Bowl, where they analyze and rethink football trends and player performances using real-world NFL data. Tackling Performance served as the theme for the 2024 Big Data Bowl, complementing the 2023 theme of Pressure Probability.

“Football is a game about blocking and tackling, and if you can’t block or tackle, you’re going to lose, no matter what. With Pressure Probability, we were able to dig into blocking metrics, but up until this off-season, we didn’t have an analogous metric for tackling,” said Abdoo.

2024 Big Data Bowl participants were given access to data from weeks 1-9 of the 2022 NFL season, analyzing the location, speed, and acceleration of all 22 players on the field, along with the location of the football during a given play. The NGS team determined that the appropriate dataset and season span would include clips of all scrimmage plays from either handoff to the end of the play, or from the moment of catch to the end of the play. The goal was to determine a metric for bringing the ball carrier down so that the NGS team could objectively quantify a player’s tackling performance.

The 2024 Big Data Bowl received a record number of applicants. Band shared, “What was key was to look at all the finalists and to extract something that they might have done well in terms of feature engineering or thought process about a stat creation. Tackle Probability was inspired by the winner but also supported by many of the other submissions. We were able to take our concept to AWS and improve that model and logic to derive the stats we wanted.”

Advancing stat creation

The process of creating the Pressure Probability stat informed Tackle Probability, due to general similarities behind each offering’s end concept. The NGS team drew from key learnings on feature engineering for Pressure Probability to get the probabilities to reach 100% and understand how to measure interactions between players, like blocker interference.

Like other NFL NGS, Tackle Probability builds on real-time data collected league-wide via radio-frequency identification (RFID) tags on players, officials, pylons, sticks, and the chains. With it, live broadcasts can now cite tackle probability and many other derived statistics for various defensive players as the game unfolds. On the flip side, a low tackle probability score (i.e., evading tacklers) indicates offensive success.

“We built this model with the idea that we could run real-time inference that could make its way to broadcasters,” noted Band.

One of the biggest changes for Tackle Probability was that model data was labeled at the play level, unlike in previous years. This new level of granularity allows the NGS team to know where and when an action happens, which is important to the model output. From there, the NGS team enlisted AWS to build out the predictive model using the insights gained from the Big Data Bowl and previous stats, and leveraged AWS compute resources to improve the model’s predictive power. They then focused on defining what to do with the model output and how to get extra derivatives from that raw output.

“AWS has been extremely flexible and great to work with in terms of the quick iteration process. They’ve been helpful in coming up with new ideas, and how to approach specific challenges,” said Abdoo. “AWS gives us a strong statistical backing and saves us time. Instead of fighting with code to try and make the right parameters, exact models, and appropriate features, we’re able to focus on using our subject matter expertise to think about the problem from a football perspective.”

Enduring teamwork

With each new advanced stat, AWS helps the NGS team uplevel data-driven insights and the fan experience. Last year, the threshold for Pressure Probability was relatively simple. For Tackle Probability, the team has fine-tuned those thresholds to better define a missed tackle. By leveraging AWS, the NGS team can scale up predictions and run many different iterations of the model. This translates to more freedom to devote engineering resources to other initiatives.

According to Band, working with AWS increases his team’s probability of success. He concluded, “Machine learning and data science is all about experimentation, and there are no guarantees. With the help of AWS, we almost certainly know that we are going to get the right outcome.”

Learn more about how the NFL is tapping into the power of AWS to advance the sport of football.