NFL Big Data Bowl - Ghost Defender

Using tracking data to learn optimal pass coverage movement and positioning

Jesse Fischer • February 21, 2021
Photo: AndrewCull

A previous post introduced Pass CNN which is a Convolutional Neural Network that predicts pass play outcomes in real-time. In this post I will show one (of many) practical applications of such work - using tracking data to learn optimal pass coverage movement and positioning.

This work was inspired by an article showing how one NBA team uses tracking data to teach optimal defender positioning.

What is optimal?

It would be misguided to think that anyone outside of coaches and players will ever know the exact play call or corresponding defensive assignments on a given play. Thus, I won't pretend to believe that my definition of "optimal" will necessarily align with those who are sitting in film study or on the sidelines. However, that doesn't mean we can't try!

What I do know is that we can use historical data to get an estimate of how successful a play will be based purely on player tracking data at each instant of a play. This is a powerful tool - as we are able to evaluate "what-if" scenarios.

What if the safety was a yard deeper at the time of throw?

What if the cornerback started moving towards the sideline a tenth of second sooner?

By holding constant the positioning and movement of all other players on the field (both offensive and defensive), we can evaluate a number of different scenarios and effectively reverse-engineer what optimal coverage might have looked like for a specific defender.

To determine defender positioning that is most optimal, I will be using my Pass CNN model can which estimates EPA based on a single "frame" of player movement data. The most optimal coverage is defined as the positioning and movement that minimizes EPA. If successful, this work has the potential to be a valuable learning tool for defenders - allowing players to see how they defended a play vs. what the model considered as optimal.

To visualize optimal vs. actual, I generate play animations with a Ghost Defender that shows where a player ideally would have been located throughout the play. You can see in the animations below how much any differences in positioning actually mattered, by comparing the real-time EPA (actual coverage) vs. Ghost EPA (hypothetical coverage). Note that Ghost EPA is the predicted EPA if the defender was removed and replaced with the Ghost Defender.

What about the coverage?

Since I don't have data on defensive coverage (i.e. Cover 1, Cover 2, etc.), I will be focusing on the optimal coverage limited to one or two defenders per play. By making the assumption that the rest of the defensive unit is generally doing what they should be, we can let the model somewhat infer a player's role based on the location of all other players. However, it is feasible to extend this work and learn optimal coverage for the entire defensive unit depending on the defensive play call (e.g. by training a different model for each defensive coverage type).

What movement is feasible?

One challenge with this work was making sure that optimal defender movement is feasible. For example, it isn't reasonable to expect players to be able to go from a standstill to top speed in a tenth of a second. One simple approach used was restricting ghost player locations to be within X yards of the players actual location. In the example below we can see what optimal positioning would have been with the Ghost Defender restricted to be within a 3-yard radius of the actual defender location.

Relative Ghost Defender


Wow! It is interesting to see the optimal defender location jump around as time advances. One benefit of this approach is that it avoids ghost defenders trying to play an entirely different role from the actual defender. This was a constant challenge, as I found that Ghost Defenders tend to be very aggressive on the blitz when near the line of scrimmage (even if it means leaving a receiver wide open).

However, this still doesn't ensure that player movement is realistic - a player can't teleport from covering one receiver to another instantaneously. Another downside is that this forces Ghost Defenders to behave fairly similarly to the actual player - which is especially limiting when evaluating a blown coverage.

To give Ghost Defenders complete freedom of movement, the ideal approach is to model physics in some capacity. Since I am not well versed on video game physics engines, I decided to use historical data to learn what type of movement is reasonable. Using a players speed in the X and Y direction from one frame, we can estimate an expected range of speed outcomes for that player in the next frame i.e. if at time T1 a player is moving S yards/sec, then at time T2 we can expect the player to be moving S +/- D (where D depends on how fast players can accelerate/decelerate).

Using this approach results in more fluid and realistic Ghost Defender movement. Note that there is an opportunity to personalize Ghost Defenders by incorporating player-specific capabilities (i.e. different players have different maximum acceleration, speed, sharpness of cuts).

Here is the exact same play but with the more realistic modeling of physics. Relative to the Ghost Defender, we can see how the cornerback overcommitted to the middle which allowed the receiver to get just enough space on the outside for a completion.

Ghost Defender X

Before looking at other examples, one last methodology note is that I found it useful to let the play develop slightly before allowing the Ghost Defender to be able to diverge greatly from the actual defender location.

Highlighted Plays

Blown Coverage

Here is an example that resulted in 29-yard Touchdown pass by Russell Wilson to a wide-open Tyler Lockett. On this 3rd and short play, both Budda Baker (#36) and Patrick Peterson (#21) played the first down line more than any receiver. Both of these players have strong instincts and were likely anticipating quick hitters in that situation. This allowed Lockett to sneak behind for an easy score. While it is hard for me to say who blew their assignment, I thought this would be an interesting play to see how Ghost Defenders would have defended.

It turns out that the Ghost Defenders also aggressively played the first down marker, however the hypothetical Budda Baker dropped back just far enough to still be able to react and give himself a chance to make a play on the ball (even if it still would have resulted in a Touchdown).

Ghost Defender X

Here is the actual video (redirect to Twitter):

Slow on crossing route

On this play we can see Mahomes complete an easy pass to Kelce for a third down conversion. This was due to the defender (#31) leaning towards the far side of the field more than he probably should have been.

On the other hand, the Ghost Defender recognized the cross route happening and was able to jump the pass, possibly resulting in a breakup if not interception. It is worth mentioning that with the tighter coverage Mahomes might have checked down to Tyreek Hill (#10) who was open in space, although Mahomes was facing pressure.

Ghost Defender X

Safety caught napping

In this last example, we look at a 37-yard completion on a play-action rollout by Matt Ryan. Julio Jones got a step on the corner after running an out route and help from the safety (#39) arrived too late. Even with most of the action moving towards the far side of the field (top of animation) the safety was slow to react. This is a good example of how play-action can help the pass, as it likely contributed to the safety freezing briefly.

Meanwhile, the Ghost Defender made a good read as the play developed and put himself in a position to challenge Julio at the ball. The positioning of the Ghost Defender turned the predicted EPA from a large positive value to a slightly negative (at the time of ball arrival).

Ghost Defender X

Feel free to follow me on Twitter if you liked this post!