Leidos researchers compete to develop the best unmanned surface vessel model
On Thursday, July 13, teams of Leidos employees battled one another out to become the grand prize winner of the six-month-long 2023 AI Palooza Competition: Pirates vs. Leidos Defenders. Each year, the AI/ML Accelerator hosts an AI Challenge to encourage employees around the company to get engaged with some of the newest AI/ML techniques through a fun, collaborative competition. Last year, the team hosted the AI Palooza Deep Racer Grand Prix Champion's Cup, which focused on building fully autonomous 1/18th scale race cars driven by reinforcement learning models.
This year, participants had to defend a simulated ship fleet against an oncoming pirate horde! Leidos’ AI/ML researchers developed tooling to connect Leidos’ Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance (C4ISR) Space and Missile Operations Simulation (COSMOS) to a Reinforcement Learning (RL) training environment. Competitors used this environment to develop a trained RL agent deployed on a COSMOS simulated Unmanned Surface Vessel (USV) capable of successfully escorting a High Value Unit (HVU) as it safely crossed the Strait of Malacca.
AI Palooza 2023 was held in partnership with AWS, Intel Corp, and Domino Data Lab – run on Intel hardware, in the AWS cloud, and distributed using Domino’s platform.
The competition was carried out in two phases. Phase 1 ran from January 27 – May 10, and phase 2 ran from May 11 – July 13. Over 60 two-person teams from Leidos UK, Leidos AU, LInC, Dynetics, Defense, Civil, and Intel Groups signed up for phase 1 of the competition. During this phase of the competition, teams developed an agent capable of directing a USV to successfully escort a HVU while two pirate vessels attacked. Performance was evaluated by the likelihood of success that the HVU made it safely across the strait. Several perfect agents were built based on different algorithms, observation spaces, action spaces, and performance reward strategies.
The competition in phase 1 was fierce, but collaborative and educational. Spencer Callicott, Software Engineering Manager in the Aerospace, Defense, and Civil branch of Dynetics said, “we learned more than we ever expected from this entire experience. I know that we can step away from it understanding that these problems are exciting and challenging, but that our approach was effective. Our customers face similar problems to this every day, and we would like to find ourselves at the forefront of solving them.”
At the end of phase 1, contestants and competition designers alike marveled at the quantity, quality, and diversity of behaviors that the trained agents could exhibit while still successfully escorting the HVU across the strait. The top-12 teams from phase 1 were selected to continue to Phase 2 of the competition. Phase 2 required teams to train an agent capable of directing a USV to escort an HVU safely across the Strait of Malacca as in Phase 1. However, Phase 2 introduced a variable number of pirates and a benign local fishing vessel. The fishing vessel acted as a distractor, requiring any successful agent to learn friend from foe, as misclassifying the fishing vessel as a pirate would not result in a successful strait crossing.
Phase 2 ended with a live, “pirate-dress-up” standoff on July 13. Phase 2 teams gathered at Leidos HQ in Reston, Va. to see a commentated, pirate-themed, single-elimination tournament. The best agents were shown defeating an increasingly difficult simulation incorporating both pirates and fishing boats. Winners of the competition, aerospace Engineers Andrew Molnar and Research Scientist Leng Ghuy said, “our strategy centered around three main points: understanding the simulation, reducing the complexity of the problem, and training to optimize agent behavior.” Their agent was the only submission to achieve perfect performance – capable of successfully defeating multiple pirates and correctly identifying and avoiding fishing boats within the simulation. For their efforts, Andrew and Leng both won Dell Workstation Precisions 5770.
This event was a showcase of companywide RL upskilling efforts and a demonstration of collaborative solutioning across multiple partner organizations with home-grown technology. It was also a chance to get together to talk about emerging techniques in AI/ML with colleagues across Leidos’ diverse business areas. Ron Keesing, Senior Vice President said, “Pirates vs. Leidos Defenders was a tremendous success on many levels, from the excitement and engagement generated to the rapid and deep upskilling in RL for dozens of technologists to the cross-organizational ties that were developed to the highly strategic technical advances that were demonstrated by the teams.”
We look forward to hosting the next AI Palooza in 2024 and continuing to increase our RL-enhanced modeling and simulation capabilities.
