Projects/Drone detection experiment pipeline
ML Systems · Data Engineering · Multi-GPU Experiments

Drone detection experiment pipeline

A recoverable experiment system for comparing detection models across heterogeneous datasets—not merely a single training run.

171,568Unified samples
3Source datasets
18 days→10–20 minVideo extraction
14/14Smoke gates

Data engineering

One task, incompatible sources

Three visible-light drone datasets arrived as VOC XML and video/XML combinations. They were normalized into YOLO and COCO representations with consistent class semantics and dataset splits.

Pipeline acceleration

Batch the expensive boundary

The original ARD-MAV process invoked ffmpeg frame by frame and was projected to take about 18 days. Exporting all frames once per video reduced the process to roughly 10–20 minutes.

Experiment operations

Make failures recoverable

The system included environment checks, small-subset smoke tests, multi-GPU scheduling, manifests, checkpoint reuse, three-seed tracking, and lightweight result aggregation.

Claim boundary

Design versus completed runs

The complete design covered six model families and three seeds. The strongest fully completed core comparison included RT-DETR-L and Faster R-CNN across three seeds each; other model results are reported separately with their own stability caveats.