Using Deep Learning To Classify Military Vehicles

Track and Monitor Transporter Erector Launchers (TELs) with Computer Vision

A transporter erector launcher (TEL) is a missile vehicle with an integrated prime mover (tractor unit) that can carry, elevate to a firing position, and launch one or more missiles. Early on, such missiles were launched from fixed sites and had to be loaded onto trucks for transport, making them more vulnerable to attack since once the enemy spotted them, they could not easily be relocated. 

The key operations and equipment used in launching a ballistic missile depend on the nature of the approach. In most approaches, the missile is delivered to the site by a truck, a train, or, at a launchpad, a dolly. The missile is erected into position. The missile is positioned either by special erectors built for the site or a crane attached to a permanent gantry. 

The high mobility of a TEL is of strategic importance to any nation's military as it enables them to transport a missile and hide it effectively from an adversary’s sight. However, there is also a downside to this, as it is exponentially more difficult to detect and locate the TELs of an adversary if the need arises. TELs are small in size (about 15-20 meters) and thus appear tiny in satellite images. Moreover, sometimes they appear to be similar to other military vehicles. They are often camouflaged using different techniques, making it much harder to detect and locate them in satellite images. Thus, here at Orbital Insight, we developed an algorithm that can help detect TELs from satellite images in an automated fashion. One key consideration is how robust the TEL algorithm is to variation in geographical regions, weather conditions, and other camouflaging techniques, which we will address later.

Data

The TEL detector was trained on Planet SkySat imagery, which is available globally. We used the red, green, and blue bands from their ortho visual product, which is pan-sharpened and orthorectified. We carefully designed the data collection process to ensure that we collected data uniformly across all regions worldwide and accounted for all seasonal variations, such as haze, cloudy, choppy waters, and snow regions. 

We generated our training data in-house by sampling scenes over various strategic military bases worldwide. A polygon was drawn around each TEL vehicle. Here are some examples of images from our dataset.

Model

We used PyTorch-based models. We experimented with various networks and eventually used Mask-RCNN for the detection. This neural network architecture accepts an image as its input and extracts relevant features from this image to output a bounding box and mask over each object of interest. 

Challenges

Camouflaging military vehicles has proven to be an extremely successful technique to hide military assets and allow military personnel to operate stealthily. Being able to operate stealthily undoubtedly confers a significant advantage in any military operation, particularly for TEL vehicles. The ability to move, relocate, and launch missiles with minimum operational delay is crucial and offers a great strategic advantage to any military. Given their strategic importance, TELs are undeniably one of the most sought-after targets by adversaries, and thus, camouflage is widely used to hide them. 

One of the most important camouflaging techniques is using color and patterns. It is used to ensure that the TELs blend into their surroundings smoothly, making them harder to detect from satellite images. 

Another major challenge was the extensive use of decoys as a means of camouflage to divert the enemy from their locations. Inflatable TEL decoys of s were found in several different locations. 

The size of the TEL was another major factor that contributed to the complexity of the task. The length of the TEL is about 15 meters with a width of about 5 meters. The SkySat satellite images used for this project were of 50cm spatial resolution after post-processing. In a SkySat satellite image of size 1000 pixels X 1000 pixels, a TEL would be about 25 pixels in length and 3-4 pixels wide. This makes it immensely difficult to detect them in satellite images and is similar to finding a “needle in a haystack.” Coupled with harsh weather conditions, such as snow and heavy clouds, and geographical complexities, such as mountain regions, the chance of missing these TELs increases significantly, and these conditions lead to more false negatives and lower recall. 

Several other military utility vehicles also appear quite similar to these TELs and are of similar size. Apart from those, there might be other stationary non-vehicle objects in nature that are similar in form and size to TELs. This can potentially result in the algorithm predicting a large number of false positives as TELs. 

The fact that the CV team was still able to overcome these challenges to a great extent and develop an algorithm to detect TELs speaks to the capabilities of our platform. In the next section, we will go over some of the techniques we experimented with to address this set of challenges. 

In the example below, we show an image with TELs that are camouflaged in the background, to some extent. Can you try and see if you are able to find where a TEL is located in the below image?

How did we address these challenges? 

It is a very well-known fact in machine learning that your model is only as good as your data. The first step we took to deal with camouflaged objects and TEL decoys was to rigorously check our data quality and ensure that no decoy object was marked as a TEL. 

We explored various cropping, resizing, and upsampling techniques to deal with the challenge of TELs appearing very small in the satellite images. Modifying model architecture to accommodate for the smaller size of input objects and varying the threshold for sampling the proposals in the network using attention modules in our model backbones, coupled with multi-scale training, helped greatly in overcoming some of the challenges of small-sized objects and adverse weather conditions. We also explored transfer learning and self-supervised representation learning techniques to boost our model performance further. 

Due to the difficulty of this problem, choosing the right set of augmentations, learning rate schedules, anchor sizes, and weight initialization techniques played a significant role in determining the final model performance. Finally, introducing additional post-processing steps to eliminate false positives gave a substantial improvement in the model's performance. 

Through several data labeling campaigns, many model tuning experiments, and innovative ideas from the computer vision team at Orbital Insight, the TEL detector algorithm obtained good performance on the evaluation set. The same approach can be used for building automated detectors for other military assets - Tanks, Mobile Erector Launchers (MELs), Ships, Airplanes, Vehicles, etc!