AI Pilot Beats Human in AlphaDogfight Trials Finale

Heron Systems’ Falco AI defeated a human F-16 pilot in five scenarios in the grand finale of the DARPA AlphaDogfight Trials. Falco bested “Banger”, a graduate of the US Air Force’s Weapons Instructor Course, in the final event of Trial 3, bringing the year-long trial program to a close with an intriguing view of things to come.

Air Combat Evolution program manager Colonel Daniel “Animal” Javorsek’s personal flight helmet, awarded to the Heron Systems team for their victory in the AlphaDogfight Trials.

The final standings of the eight teams participating in the AlphaDogfight Trial 3 are:

  1. Heron Systems
  2. Lockheed Martin
  3. Aurora Flight Systems
  4. PhysicsAI
  5. SoarTech
  6. Georgia Tech Research Institute
  7. EpiSci Science
  8. Perspecta Labs
The Lockheed and physicsAI competitors fighting in the semi-finals on Day 3.

The fight against the human F-16 pilot is the culmination of the three day trial. On the first day, the competing AIs fought against AIs developed by Johns Hopkins Applied Physics Laboratory that simulated a variety of basic fighter maneuvers (BFM) engagement scenarios. Day 2 saw the AIs compete against each other in a “round-robin” match-up of multiple engagement scenarios. The four highest-scoring teams proceeded to Day 3, with the scoring being a combination of Day 1 and Day 2 scores. The AIs’ performance against the Johns Hopkins Applied Physics Laboratory AIs counted towards 20% of their final score, with the remaining 80% from their Day 2 performance.

One of the “round-robin” matches on Day 2. All competitors run through 20 test cases per match against a competitor.

Heron Systems’ Falco dominated Day 1 and Day 2, with its extremely aggressive pursuit of shots on its opponents, up to head-on charges, allowing it to quickly defeat opponents, sometimes in a single pass. Lockheed’s AI performance significantly improved in Day 2’s scenarios, jumping to second place in Day 2 compared to its sixth place finish in Day 1. The gap between PhysicsAI and Aurora Flight Systems’ AIs closed on Day 2, with Aurora Flight Systems finishing in third place, a single point ahead of PhysicsAI. With the scores combined, PhysicsAI remained in third place into Day 3, with Aurora Flight Systems closely behind it.

Perspecta Labs’ AI fleeing from APL’s Alpha Mav AI on Day 1. Perspecta attempted the same tactics on Day 2, with similar lack of success.

AlphaDogfight Trial 3 was originally intended to be held in April in Las Vegas, but the COVID-19 pandemic resulted in the delay of Trial 3 to August, with the event livestreamed on Zoom instead from the Johns Hopkins Applied Physics Laboratory in Laurel, Maryland. Overwhelming demand resulted in registrations closing early, with around 10,000 viewers from 93 different countries having registered by the time registrations closed. DARPA announced yesterday that the finale with the winning AI fighting a human pilot would be livestreamed on DARPA’s YouTube channel, owing to the continued demand. Following their victory Heron Systems Inc. held a streamed Q&A session on their own YouTube channel.

A screenshot of Falco and Banger’s second match. Banger’s view flashes red when taking damage, and flashes blue when dealing damage.

While Falco dominated the five matches against Banger with the same aggression as it did its fellow AIs, there are some caveats to the tests that should be kept in mind. All AIs participating in the trials have perfect state information, something that could only be partially simulated for Banger and unlikely to be the case in a “real world” fight. Additionally, all scenarios in Trial 3 were one-on-one guns-only within visual range combat. Regardless, this is hardly the end of the road for Falco or the other AIs. The VR simulator rig used by Banger will be moved to Nellis Air Force Base in the future, allowing for other pilots to test themselves against the AIs – or even provide a rematch.