EVPropNet: Detecting Drones By Finding Propellers For Mid-Air Landing And Following
Aerial robots are ubiquitous nowadays and can be used for various applications in diverse fields. At the same time, the use of malicious drones has also increased. Therefore, automated drone detection systems are necessary.
A recent paper proposes to detect drones from the most ubiquitous part of a drone – the propeller.
Classical imaging cameras cannot detect propellers because they rotate at high speed. Fortunately, modern event cameras output per-pixel temporal intensity differences caused by relative motion with microsecond latency. For the detection, the fact that propellers move much faster than any other part of the scene is utilized. A deep neural network is trained on the simulated data. It generalizes to the real world without any fine-tuning or re-training. It can be used for tracking and following a drone or for landing on a near-hover quadrotor.
The rapid rise of accessibility of unmanned aerial vehicles or drones pose a threat to general security and confidentiality. Most of the commercially available or custom-built drones are multi-rotors and are comprised of multiple propellers. Since these propellers rotate at a high-speed, they are generally the fastest moving parts of an image and cannot be directly “seen” by a classical camera without severe motion blur. We utilize a class of sensors that are particularly suitable for such scenarios called event cameras, which have a high temporal resolution, low-latency, and high dynamic range.
In this paper, we model the geometry of a propeller and use it to generate simulated events which are used to train a deep neural network called EVPropNet to detect propellers from the data of an event camera. EVPropNet directly transfers to the real world without any fine-tuning or retraining. We present two applications of our network: (a) tracking and following an unmarked drone and (b) landing on a near-hover drone. We successfully evaluate and demonstrate the proposed approach in many real-world experiments with different propeller shapes and sizes. Our network can detect propellers at a rate of 85.1% even when 60% of the propeller is occluded and can run at upto 35Hz on a 2W power budget. To our knowledge, this is the first deep learning-based solution for detecting propellers (to detect drones). Finally, our applications also show an impressive success rate of 92% and 90% for the tracking and landing tasks respectively.
Research paper: Sanket, N. J., Deep Singh, C., Parameshwara, C. M., Fermüller, C., de Croon, G. C. H. E., and Aloimonos, Y., “EVPropNet: Detecting Drones By Finding Propellers For Mid-Air Landing And Following”, 2021. Link: https://arxiv.org/abs/2106.15045
Link to the accompanying video: https://prg.cs.umd.edu/EVPropNet