A new take on tackling: NFL Next Gen Stats and AWS bring data to the gridiron

In the high-stakes world of professional football, every fraction of a second counts. A defender’s split-second decision can be the difference between a game-changing tackle and a touchdown. For years, the National Football League (NFL) has sought to quantify these crucial moments, but the complexity of measuring the different factors contributing to a made or missed tackle made this a challenging task. Enter Amazon Web Services (AWS) and the NFL’s Next Gen Stats (NGS) team, who together embarked on an ambitious project to build Tackle Probability, a new AI-powered model that changes how we understand and analyze tackles in football.

The journey began with a spark of inspiration from the NFL’s Big Data Bowl, an annual competition that challenges data scientists to push the boundaries of football analytics. Among the submissions was an intriguing model designed to predict the likelihood of a successful tackle at any point during a play. While innovative, this model served as merely the starting point for a more ambitious endeavor. The larger goal was to develop a tackle probability model that could analyze massive datasets in near real-time, offering deep, real-time insights into one of the most critical moments in a game.

The first hurdle the AWS and NGS teams faced was the sheer scale of the data available to work with. The original model from the Big Data Bowl was trained on just eight weeks of data from a single season. To create a truly comprehensive model, the team needed to expand this dramatically. They set their sights on using a full season’s worth of data for training and another full season for testing. This expansion presented a significant engineering challenge—how to efficiently process and analyze such a massive dataset without sacrificing speed or accuracy.

This is where the power of AWS cloud infrastructure, particularly the Amazon SageMaker suite of machine learning (ML) tools, came into play. AWS engineers leveraged scalable computing resources available with SageMaker to handle the increased data volume, but they knew that raw processing power alone wasn’t enough. They needed to approach the problem with smart and creative thinking, along with extensive knowledge of football mechanics.

One of the key focus areas for AWS engineers was feature engineering—the process of identifying and creating new data points that can improve a model’s predictive power. This wasn’t just a matter of crunching numbers; it required a deep understanding of the game of football itself. The team analyzed game footage, consulted with NGS data analysts, and brainstormed new ways to quantify the complex dynamics of a tackle.

To support this exploration, AWS developed a framework using SageMaker and MLflow that allows the NFL to track experiments efficiently. This system enables rapid iteration, where features can be added or removed easily, and different model architectures and parameters can be explored, documented, and deployed. The framework can deploy any of these experiments into production-ready Docker containers within minutes, streamlining the path from research to production.

These efforts led to the development of approximately 15 new features across 12 different categories. Some of these features were relatively straightforward but surprisingly impactful. For instance, the engineering team looked at the ball carrier’s distance to key points on the field like the sideline, end zone, and line of scrimmage. While these might seem like obvious factors to consider, they were not explicitly included in previous models. The team was almost surprised by how much these simple spatial relationships improved their predictions.

Image 1: SageMaker GroundTruth overlay of predicted tackle probability as the play unfolds.

Other features delved into more complex on-field dynamics. AWS engineers introduced new methods to measure how blockers impede or obstruct defenders and how the coordinated efforts of multiple defenders can increase the likelihood of a tackle. The team also developed a novel feature to compute blocker interference scoring by creating a mechanism that analyzes the positions of offensive blockers and considers the number of offensive blockers between the defender and the ball carrier. They even explored how a player’s weight might influence tackling dynamics, though this ultimately proved less significant than expected.

Image 2: SageMaker GroundTruth overlay of predicted tackle probability as the play unfolds.

One particularly innovative approach involved looking at how defenders “encircle” the ball carrier from different angles. By considering factors like the number of nearby defenders and their relative speeds to the ball carrier, the model gained a more nuanced understanding of how tackle opportunities develop. This kind of feature engineering required not just technical skill, but a genuine understanding of football strategy and player behavior.

As the team refined features and built increasingly complex models, they faced another significant challenge: how to efficiently test and validate their work. With millions of data points to process, running predictions could take weeks using traditional methods. This is where AWS cloud infrastructure played a key role.

The team implemented SageMaker Batch Transform for large-scale inference, allowing them to run predictions on an entire NFL season’s worth of data—over 15 million individual predictions—in about an hour. This represented a staggering improvement in computational efficiency, compressing a month-long process into less than 60 minutes. This dramatic speedup wasn’t just about saving time; it fundamentally changed what was possible in terms of model evaluation and refinement. The ability to quickly test predictions against a full season of data allowed for more rigorous validation and helped eliminate potential sampling biases.

As the model grew more sophisticated, the team explored various architectures, focusing primarily on tree-based methods like XGBoost and Random Forest, model architectures that can generate the low-latency predictions required during live broadcasts. They employed both manual and automated feature selection techniques, including Sequential Backward Selection, to identify variables with the most impact. This systematic approach helped them narrow down their extensive list of potential features to a more manageable and effective set.

The final model ended up using 20 features, 14 of which were newly developed by the AWS team. When compared to the baseline model (trained and tested on the same expanded dataset), the new model showed improvements across all key performance metrics. But the team knew that raw performance numbers weren’t enough—they needed to create something for use in real-world broadcasting scenarios.

This led to the development of derivative metrics like “tackle attempts” and “missed tackle attempts.” The AWS team worked closely with NGS analysts to define precise criteria for these events, balancing the need for accuracy with the desire for real-time or near-real-time reporting. They aimed for a missed tackle rate of around 15-20%, in line with historical data from other sources.

One of the most exciting aspects of the project was the development of a custom user interface using SageMaker Ground Truth. This tool allowed NGS analysts to review plays visually, seeing the model’s tackle probability predictions overlaid on video of the actual play. This visualization capability proved invaluable for both validating the model and identifying edge cases where it needed improvement. It also highlighted situations where the raw tracking data might not capture all the nuances visible to the human eye, such as when a defender and blocker are locked together in a way that should lower tackle probability.

As the project neared completion, the team faced one final challenge: ensuring the model could perform in real-time game scenarios. In a major engineering feat, they optimized the implementation to provide predictions with sub-100 millisecond response times, making it feasible for use during live broadcasts. This required careful balancing of accuracy and speed, pushing the limits of what was possible with current technology. With this breakthrough, the model can be effectively used in real-time broadcast scenarios to provide instant insights during live games.

To ensure reliability and performance, the AWS team implemented automatic unit testing during the build stage. This catches potential inference slowdowns or other issues before deployment, saving considerable debugging time. Additionally, we conducted stress tests using Locust, which simulates swarms of users hitting the container. These tests confirmed that the container, when deployed on an Amazon EKS cluster, could scale up to serve full broadcasts across multiple teams while maintaining strict latency requirements.

Throughout the entire process, the collaboration between AWS engineers and the NGS team was crucial. The AWS team brought expertise in cloud computing and machine learning, while the NGS analysts provided deep football knowledge and helped ensure the model’s outputs would be meaningful and useful in real-world scenarios. This interdisciplinary approach was key to the project’s success, bridging the gap between cutting-edge technology and the practical needs of sports broadcasting.

As the 2024 NFL season kicks off, fans tuning in might not recognize the sophisticated data science powering the stats they see on screen. Behind those numbers lies months of intensive work, leveraging some of the most advanced cloud computing and machine learning technologies available. The tackle probability model represents more than just a new stat—it’s a testament to what’s possible when innovative engineering meets deep domain expertise.

But the story doesn’t end here. The team sees plenty of room for further refinement, from addressing niche scenarios to exploring new model architectures. And the techniques and tools developed for this project have potential applications far beyond football. Any domain dealing with complex, time-series data and the need for rapid, large-scale predictions could benefit from similar approaches.

In the end, this project demonstrates the transformative power of cloud computing and machine learning in sports analytics. By leveraging AWS engineering expertise and technology stack, particularly SageMaker, the NFL has taken a significant step forward in its ability to quantify and analyze the game. As we look to the future, one thing is clear: the intersection of sports and technology will continue to push the boundaries of what’s possible, giving fans, coaches, and players new ways to understand and appreciate the game they love.