A critical backup for a driver’s most important tool
In a recent joint NATO exercise in Germany, the drivers of several U.S. Humvees received pinpoint electronic navigational guidance as they drove through fields, forests, and villages. Of course, drivers all over the world get that guidance every day from their phones. But there was an unusual twist to the precise location information being fed to the Humvees on that day: It was produced without the aid of GPS, long the essential ingredient of virtually all electronic navigation systems.
Bringing that sort of unprecedented location accuracy to military ground vehicles of all types without relying on GPS has been the mission of a Leidos team of engineers in Huntsville, Alabama. Working with whatever information can be gleaned from cameras, compasses, and other sensors readily available on the vehicles, the group has developed software called Assured Data Engine for Positioning and Timing (ADEPT). ADEPT can weave the data together into an accurate navigation solution critical to mission safety and success — even when a GPS signal isn’t available.
“GPS is the most trustworthy location information we have,” says Troy Mitchell, the Leidos program manager who directed the project. “Providing something that can work nearly as well when GPS is denied was a tough problem to solve.”
Why would GPS disappear? Anyone who has driven through a big city knows what it’s like to periodically lose the signal in an “urban canyon” — that is, a street lined with tall buildings. That can be a real annoyance to a civilian. But to soldiers, Marines, and Special Forces driving through an unfamiliar urban combat zone overseas, losing position and navigation data can be a serious threat. Even more worrisome, opposing forces may develop the capability to jam, spoof, or even switch off GPS signals anywhere in the world, on any terrain.
ADEPT finds ways to wring accurate locations from the information on and around vehicles. One key is in the accurate interpretation of images from the video cameras that are fast becoming standard equipment on military vehicles. The basic idea seems simple enough: Just as a person can look around and figure out where they are by matching landmarks to a map, software can be taught to do much the same. The question was, could it be done on the fly in remote, hostile territory, with GPS-like accuracy on a vehicle with limited computing capacity? In a quest to find out, the Leidos team enlisted three machine learning techniques to turn video images from a moving vehicle’s camera into a map location. Spoiler: It works.
SLIM, SLAM, and Loops
Street-level image matching (SLIM) relies on third-party databases of 3D global maps that can be used to render landscapes as they would look from the ground, rather than being limited to the birds-eye view of most map databases. With SLIM, the software is looking for a match between the view from the vehicle’s camera and the database’s 3D view of nearby terrain. “It’s kind of like driving around a strange city and comparing what you see to postcards of the area,” says Dr. Tom Grieve, a Leidos senior engineer who helped lead the computer-vision aspects of the system. Because the images in the 3D databases are geotagged with longitude and latitude, getting a match provides the vehicle with an absolute location fix on the map, which is key to minimizing the drift that otherwise accumulates from inertial sensors.
When a vehicle is in an area where 3D database images aren’t available, ADEPT uses SLAM, or Simultaneous Localization and Mapping. With SLAM, the system examines the relative speeds at which different objects and features captured by the camera, such as trees, hills, or buildings, move across the image frame. “When you’re driving around, things that are close look like they’re moving faster than things that are further away,” explains Grieve. That motion information can be enlisted by the software to calculate the relative locations of surrounding features. Tracking those relative locations over time can help determine how far the vehicle has moved since the last time an absolute location fix was found.
Finally, the system can rely on “loop closure” in cases where an initial match to a map location can’t be made. It’s a technique that takes advantage of the fact that the system is continuously recording key features that the vehicle camera sees. Think of it as the image’s thumbprint. If at any point in the future the system identifies a matching thumbprint — in other words, if the vehicle ever completes a loop or another vehicle on the network traverses the same area later — the system can correct any inertial drift that has accumulated in the meantime.
Contributions from multiple vehicles can be combined to assemble a larger, more detailed, more accurate map that all the vehicles can share. This “distributed” loop closure is an example of a collaborative navigation technique developed by Leidos.
In addition to camera data, the Leidos system can also monitor the vehicle’s speedometer and magnetometer (essentially a compass) to supplement the image data and better keep track of location between exact fixes. “The vehicle’s speed and direction of motion are the two biggest sources of error in figuring out where the vehicle is,” says Mitchell. “Using data from those two instruments helps cut the errors down.” He notes that many military ground vehicles also carry inertial navigation units, which use accelerometers and gyroscopic devices to sense motion and create platform reference states that are continuously updated with the sensor fusion capability of the ADEPT software.
Bringing GPS backups to military vehicles
The multiple information streams coming in from cameras and other instruments are integrated by ADEPT to get the best mix of accuracy and reliability. Along the way, the software provides calibration for misaligned instruments — such as a compass that consistently reads one degree off — and adjusts for noise and other factors, to generate a single location estimate that is as reliable as possible.
The resulting fix from the system can be fed to the vehicle’s existing navigation screen, so there’s no learning curve or adaptation needed on the part of military personnel.
“We take our prototypes out into the field with soldiers, Marines, and special ops, so we can learn more about what they need,” says Mitchell. “They made it clear they want to stick with what they’re used to, and they don’t want any new boxes in their vehicles.” To that end, the Leidos software often runs on the existing platform computer and works with onboard cameras and sensors.
That means the system could kick in to replace a lost or degraded GPS signal on a critical mission without anyone in the vehicle even realizing it. After all, the highest compliment sophisticated military tech can get in the field is to be counted on without anyone having to give it a second thought.
