Thermal Drone Detection

Drones in thermal video are often only a few dozen pixels across, and birds or aircraft can look much the same. How far does a pretrained detector get after fine-tuning on public thermal video, compared with a simple hot-spot detector, and what happens on footage from a different camera and site?

A data pipeline for Anti-UAV300 (infrared videos only) and for the Svanström et al. 2021 multi-sensor drone dataset. It checks each video's frame count against its labels, extracts frames, and keeps the second dataset for evaluation only. Five videos whose labels did not line up with their frames were excluded before any model was run.

While splitting the data I found that the official Anti-UAV train and validation sets share 26 of their 27 source videos (a sequence's name gives its source video). Splitting by clip would let nearly identical scenes appear on both sides, so I merged the two sets and re-split them by source video: 22 source videos for training and 5 for validation. The official test set (91 sequences from 10 other source videos) was kept as it was.

The training-free baseline marks hot or cold spots with a morphological top-hat filter and boxes them. Its kernel size, threshold and a mask for the text the camera burns into each frame were chosen on validation data only.

The model is torchvision's Faster R-CNN with a ResNet-50 feature pyramid (BSD-3-Clause), fine-tuned for four epochs on an Apple M2 with smaller anchors (from 8 px), 320-pixel crops and random down-scaling. I chose it over Ultralytics YOLO because YOLO is AGPL-3.0, which would place copyleft obligations on this MIT-licensed project.

A box counts as found if it overlaps a labelled drone with IoU ≥ 0.5. AP@0.5 is the area under the precision–recall curve. The false-alarm rate is the share of drone-free frames that get at least one box above the threshold.

Anti-UAV test: 9,582 frames from the 91 test sequences, 8,429 drone boxes and 1,153 drone-free frames. Recall by target size adds 340 frames with small targets, because such targets are rare in this dataset.

Svanström: every fifth frame, never used for training or tuning. For the bird, aeroplane and helicopter videos I report two measures: how often a frame raised any drone alarm, and, on frames where the object is labelled, how often the drone box sat on that object.

Brackets are 95% intervals from a video-level bootstrap with 1,000 resamples. The resampling unit is the test sequence on Anti-UAV and the video on Svanström.

On the Anti-UAV test set, fine-tuning raised AP@0.5 from 0.139 to 0.885 and cut false alarms on drone-free frames from 99.5% to 3.5%. The training-free baseline raised an alarm on all but 6 of the 1,153 drone-free frames.

Recall by target size (√(w·h)) was 71.8% for 8–16 px (262 / 365), 78.2% for 16–32 px (1,537 / 1,965), 86.7% for 32–64 px (5,440 / 6,272) and 100% above 64 px (150 / 150). It found none of the 17 targets under 8 px.

On the dataset it never saw, drone recall fell from 84.9% to 48.6%, and to 17.6% for drones more than 60 m away. A score measured within one dataset did not carry over to another camera and site.

Most alarms in bird, aeroplane and helicopter videos were not on the labelled object. 23–52% of their frames raised a drone alarm, but the drone box sat on the bird or aircraft itself in only 3–19% of labelled frames. Some of the rest may be unlabelled objects or boxes short of IoU 0.5; I did not check them one by one.

I set the rules: public datasets only, video-level splits, thresholds chosen on validation data only, and every number taken from a run. For the first write-up I stopped after one epoch and scored reduced samples; the full four-epoch run with full evaluation has since replaced every number on this page.

Claude implemented the data pipeline, baseline, training and evaluation in bounded steps on my instructions. I reviewed its output and keep responsibility for the claims on this page.

Only 17 test targets were under 8 px, and neither method found any of them (0 / 17), so there is no conclusion about very small drones. Some missed small-target labels sit at the edge of the frame with no drone visible, so part of these misses may be labelling errors; I did not check them one by one.

The baseline's chosen setting has k = 1.5, the lower edge of its search grid, so the baseline may be slightly understated.

The Anti-UAV test set comes from only 10 source videos. Its 91 sequences, the bootstrap units, are drawn from those 10, so the intervals are wide, and because sequences from the same source video are correlated the real uncertainty may be wider still.

Training did not run in one go. Epoch 1 used the original setting (2 data-loader workers, batch 8); epochs 2–4 resumed from the epoch-1 checkpoint and optimizer state with a low-memory setting (no workers, 2 images × 4 gradient-accumulation steps, effective batch 8). PyTorch's global random state was not restored, so the detector's random region sampling differs from an uninterrupted run, and retraining on MPS can change the last decimals.

Both datasets come from specific cameras and sites, and the second one was recorded in daylight only. These results say nothing about other sensors, weather or operational use.

The failure examples below were made with the first (1-epoch) model and have not been regenerated with the full-run model.

Code, configuration and the full-run results (results/full/: the comparison table and the metric files for both methods) are in the public repository, along with the first run's results and the failure gallery. Every result file records a hash of the configuration, the SHA-256 of each dataset archive and the commit of my private working repository at run time; the public repository starts from a fresh history. The seed is fixed at 0.

Datasets: Anti-UAV300 (repository under the MIT License) and the Svanström et al. multi-sensor drone dataset (data released under CC0 1.0; article under CC BY 4.0). Model weights and raw videos are not published.

Jiang et al., Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking (arXiv:2101.08466) ↗

Anti-UAV dataset repository (MIT License) ↗

Svanström, Alonso-Fernandez & Englund, A dataset for multi-sensor drone detection, Data in Brief 39, 107521 (2021) ↗

Svanström et al. drone detection dataset, Zenodo (CC0 1.0) ↗