| UAV | |
Intelligent, automated, rapid, and safe landmine, improvised explosive device and unexploded ordnance detection using Maggy
In this study, a small-scale customised drone – the so- called Maggy – was developed to simplify and automate the procedures of cleaning explosive devices. We present here the first part of the paper. The conculding part of the paper will be publsihed in March issue. |
 |
|
 |
|
 |
|
 |
|
 |
|
 |
|
 |
|
 |
|
Abstract
Detecting and clearing legacy landmines, Improvised Explosive Devices (IED), and Unexploded Ordnances (UXO) using a force made up of humans or animals is extremely risky, labour-and time-intensive. It is crucial to quickly map millions of buried landmines/IDE/ UXO, and remove them at a reasonable cost to minimise potential risks and make this labour-intensive task easier. Using unmanned vehicles and robots outfitted with various remote sensing modalities appears to be the ideal way to carry out this task in a non-invasive manner while employing a geophysical investigative method. In this study, a small-scale customised drone – the socalled Maggy – was developed to simplify and automate the procedures of cleaning explosive devices. It was instrumented with innovative intelligent automated techniques and magnetometer sensor technologies. Maggy’s performance was assessed in field tests conducted in Latvia and the United Kingdom. The outcomes, obtained in the open-air minefields and the benchmark assessments, verify the viability of the technologies, methods, and approaches integrated into Maggy for the efficient and economical detection of legacy landmines and IDE/UXO. This research provides the related research community with fundamental design and implementation parameters (e.g. flight speed, flight altitude) in building and using magnetometer-integrated Unmanned Aerial Systems (UAS). The improved versions of the developed easy-to-use compact technology are aimed to be deployed by humanitarian demining teams to expedite their clearing operations safely and efficiently.
I. Introduction
Detecting and clearing legacy landmines (anti-tank (AT) and anti-personnel (AP)), Improvised Explosive Devices (IED), and Unexploded Ordnances (UXO) using a force made up of humans or animals is extremely risky and labour- and timeintensive [1]. When these explosives come into contact with, are near to, or are in the presence of a person or vehicle, they explode. In particular, AP landmines cause long-term casualties and psychological effects by mutilating, rather than killing. More than 1,000 deminers have lost their lives or suffered injuries while performing demining operations between 1999 and 2012 [2]. All around the world, there are approximately 100 million buried landmines [3] due to the low-cost manufacturing [4] and simplicity of deployment across wide regions. 61 states worldwide are severely impacted [5] by the slow demining process [6]; these include, but are not limited to, Croatia, Bosnia and Herzegovina, Serbia, Afghanistan, Montenegro, Libya, Syria, Iraq, and most recently, the wartorn regions of the west of Ukraine and Azerbaijan. By the end of 2005, Bosnia and Herzegovina declared that there was a possibility that over 4% of their territory was contaminated with landmines [7]. In 1997, two years after the war ended, 23% of Croatian territory was thought to be mine-suspected [7]. 10.413 people in Colombia, one of the nations most affected by landmines worldwide, lost their lives to landmines between 1990 and 2013 [8]. Over 35,000 amputees in Cambodia have been impacted by a landmine explosion [2]. The average number of people killed or maimed annually is 26,000 [9] and 80% of this figure is children [5]. Ten mines are placed for every mine removed, despite recent efforts to reduce their use [10]. The precise locations of legacy landmines that have been buried are unknown, and landmines can shift slightly depending on the features of the land and the time they were buried. Using conventional methods to remove millions of landmines/IDE/UXO would take more than a century [11] with potential risks and high costs [12], which will have a longterm, significant impact on these nations in a variety of ways. Their presence continuously puts communities in danger, obstructs economic growth, and makes it difficult for infrastructure, agriculture, and resettlement to have safe access to land. The development of a landmine/UXO/ IDE detection system that is quick, safe, and economical is urgent. Land-based vehicles face a number of challenges, including accurate navigation over rough terrain despite being supported by various mechanisms like wheeled, legged, and dragged robots [13]. Furthermore, it takes a while to scan larger terrain with those slow, heavy vehicles. Autonomous drones have recently been deployed to accomplish a diverse range of missions (e.g. logistics [14], smart cities [15], agriculture [16]), due to their efficient and effective use. Drones can expedite surveying and provide better access to challenging terrain with tough and hard-to-reach topography and thick vegetation [17], [18], [19]. Unmanned Aerial Vehicles (UAVs) suited to covering a large area for the purpose of easing labour-intensive mine clearance have been used in numerous studies with different detection approaches. These studies are analysed in Section II.
Magnetometers consume very little power in addition to their affordable and lightweight features and drone applications can benefit from using magnetometers in a diverse range of applications efficiently and effectively. This work, by developing a magnetometer-integrated Unmanned Aerial Systems (UAS) has been focused on landmine/UXO/IDE detection, primarily, for supporting the humanitarian clearance challenges and constraints around the world – such as the need to operate in unforgiving, undulating terrain, which may be overground with vegetation. The contributions are listed below to make the novelty of this paper clear.
A bespoke, low-cost, small footprint, easy-to-use, and autonomous robotic drone – the so-called Maggy (Figs. 14, 15) – integrated with magnetometer sensor modalities (Fig. 10) was developed to detect landmines/IDE/UXO locations rapidly and safely. Low mass, small size, and lightweight Maggy with low energy consumption is capable of inspecting fields at low altitudes through pre-programmed routes with extreme height precision and terrain following mode for revealing the probable landmine/UXO/IDE spots.
A tablet/smartphone application (Fig. 16) was developed and integrated with Maggy to i) manage Maggy, ii) process real-time data streaming from Maggy to locate landmines/IDE/UXO, iii) perform detailed survey analysis considering varying magnetic fields (MF), and iv) communicate with the landmine/ UXO/IDE clearing team for reporting exact landmine/UXO/IDE locations.
The developed small, lightweight and robust aerial platform can be carried in a backpack and rapidly deployed by humanitarian demining teams in supporting their humanitarian landmine/UXO/IDE clearance activities safely and efficiently.
This research provides the related research community with fundamental design and implementation parameters (e.g. flight speed, flight altitude) in building and using magnetometer-integrated UAS.
The rest of the paper is organised as follows. The literature survey is conducted in Section II. The developed approaches and techniques in this study are explored in Section III. The results within the experimental setup are presented in Section IV. Results and findings are discussed in Section V. Section VI draws conclusions followed by the limitations in Section VII. Finally, Section VIII provides directions for potential future works.
Related Works
Metal detector technologies, electromagnetic (e.g., ground-penetrating radar (GPR), microwaves, nuclear quadrople resonance (NQR), infrared (IR), electrical impedance tomography, X-ray backscatter, neutron methods, sound and ultrasound), acoustic/seismic, biological (e.g., rats and dogs, bacteria, bees, antibodies, chemical methods), mechanical methods (e.g., prodders and probes, mine-clearing machines) are the main non-invasive methods employed in landmine detection [10]. Among these, metal detectors are the most commonly used tools for detecting landmines in humanitarian demining [7]. The capabilities and limitations of metal detectors are analysed by Dieter et al. [7] for determining which detector is appropriate to be used under what circumstances. The ever-evolving technology of landmines poses a significant obstacle to clearance efforts [20]. Existing metal detectors require the user to be physically close to the scan area, and that presents a real risk of injury or fatality when the area has emplaced ordinance either buried or scattered on the surface. Such systems tend to give an audio warning when a detection is made, and it is not recorded or geostamped. Detecting new landmines is more difficult because they contain fewer or no metals [2]. Stated differently, there are numerous varieties of landmines composed of diverse materials, including plastic, glass, wood, and metal, and they come in a range of sizes [21], most of which are undetectable by conventional electromagnetic-induction (EMI) methods used in metal detectors.
A number of other diverse approaches have been employed to mitigate the shortcomings and constraints of the metal detectors. The use of GPR seems a viable option to support metal detectors and increase the detection accuracy of a demining system [9], [22], [23], [24] where it can detect a wide range of landmines, especially, in detecting non-metallic objects at depth, even though it is susceptible to various localised ground inhomogeneities and surface roughness [22], [20]. In addition to being sensitive to local inhomogeneities of the ground, the small electromagnetic (EM) radar cross sections for non-conducting materials make it challenging to detect buried explosives made of dielectric or polymer-based materials (plastics) [25], [26]. Moreover, regarding sensing capabilities, highpriced GPR systems have limitations due to strong random clutter at rough airsoil interfaces [27], the size of targeted objects (<10 cm) [28] and soil moisture and flight height [29]. To overcome these deficiencies, there have been numerous attempts to employ various other sensor modalities as mentioned earlier different from metallic detectors and GPR to reduce the false alarm rate (FAR), increase the chance of detection, and expedite the landmine/UXO/IDE clearing operations safely. Every technique used in these attempts has shortcomings. For instance, Lihan et al. [21] and Ishikawa et al. [30] assess dual sensor approaches that make use of both EMI and GPR sensors to compare the effectiveness of dual sensors and metal detectors. These approaches are particularly effective in differentiating between landmines and metal fragments and extending the detectable range in the depth direction. Donskoy et al. [31] use remote measurements of soil surface vibration (using laser or microwave vibrometers), processing of the measured vibration, and vibration (using seismic or airborne acoustic waves) of buried objects to extract the “vibration signatures” of mines.
Thanks to cyber-physical systems (CPSs) and enhanced Artificial Intelligence (AI) techniques, recent years have seen an increase in the intelligence of the “everyday things” in our environments considering Internet of Everything (IoE) [32], [33] enabling them to make decisions with an increasing degree of autonomy and little to no help from humans, leading to the development of advanced robotics systems. In addition to using different types of sensor modalities, there are various initiatives to speed up the demining process and prioritise safety using robotic systems. For instance, Aoyama et al. [3] propose a land vehicle robot with a mine detector; Sun and Li [23] propose a mine detection using a land vehicle on which a forwardlooking GPR (FLGPR) is mounted. In particular, to more quickly detect landmines on larger fields, vision-based remote sensing (VBRS) modalities are becoming more and more popular as a solution to the drawbacks of the currently in use of off-the-shelf conventional techniques. These methods are founded on various physical principles, e.g., vapor/builk detection, electromagnetic detection, and optoelectronic imaging [34]. Nonetheless, a number of factors, including the type of soil, weather, lighting, and ambient conditions, must be taken into consideration when applying these techniques successfully [34]. More specifically, over the past 20 years, spectral remote sensing technology has made great strides and is now being utilised more and more in lab-scale applications (such as forensic, biomedical, industrial, biometric, food safety, and pharmaceutical process monitoring and quality control) [35]. Increased and sustained agricultural yields, water resource management, food safety and quality evaluation, disease diagnosis, artwork authentication, forensic analysis of disputed documents, military target detection, and counterterrorism have all benefited from the use of hyperspectral imaging [36]. By exploiting this technology, Banerji and Goutsias [37] suggest combining an aerial minefield imaging system with multispectral (multiple wavelengths) sensors as part of a morphological approach to automatic mine detection. Anderson et al. [38] analyse the multispectral photos to look for landmines on the basis of histograms. Differentiating the thermal properties of the soil and the buried objects is how the detection is made [34]. Thanh et al. [34] suggest a finite-difference approximation of generalised solutions to the thermal model as a 3-D linear forward thermal model for buried landmines. Among the technologies in use, the dynamic thermal infrared technique (IR images of the soil surface obtained at multiple time instants) appears to hold promise for the detection of non-metallic landmines that are shallowly buried and for differentiating them from other buried objects by utilising the differences in thermal properties between the buried objects and the soil [39] [34], [39]. In other words, the existence of buried objects influences the soil’s ability to conduct heat, leading to variations in soil temperature above the objects compared to areas that have not been disturbed; an IR imaging system situated above the soil area can measure this temperature signature [39].
The use of UAVs is clearly suited to covering larger minefields without the danger of triggering landmines/IDE/UXO during humanitarian clearing activities. The incorporation of UAS equipped with various sensor modalities into clearance operations has recently become popular. García Fernández et al. [40] propose a synthetic aperture radar imaging system for landmine detection using a GPR integrated with a drone. Measurements in controlled and real-world scenarios validate the algorithms and the UAV payload, demonstrating the viability of the suggested system. Mine Kafon integrated both a GPR and a metal detector with an aerial vehicle as shown in Fig. 1. García Fernández et al. [41] suggest using an aerial Synthetic Aperture Radar (SAR) imaging system to obtain complete three-dimensional (3D) radar images from below the ground. Schartel et al. [42] carried out airborne landmine detection with a circular synthetic aperture radar. Garcia Fernandez et al. [43] analyse airborne multi-channel ground penetrating radar for landmine/UXO/IDE detection. The use of GPR systems, with their large size and heavy weight, on UAS is extremely restrictive, especially, on lightweight drones with smaller payloads (Fig. 5). Badia et al. [4] suggest a blimp-based UAV outfitted with a widely tuned metal-thin oxide chemo-sensor through the use of a bioinspired detection architecture where employing trained animals is still one of the most widely used techniques for explosive detection. Colorado et al. [13] suggest a UAV-based system that recognises and processes images of partially buried landmine-like objects.
According to a market research report by MarketsandMarkets, the global magnetometer market was valued at around USD 2.44 billion in 2023 and was projected to reach USD 4.34 billion by 2032, growing at a compound annual growth rate (CAGR) of around 6.60% during the forecast period [44]. These figures indicate the market’s significant size and potential for expansion. The active detection of small UXO by measuring electromagnetic responses is analysed in [45] using a magneto-inductive sensor array, in [46] using broadband electromagnetic induction sensors, and in [47] using fluxgate sensors. The detection and classification of subsurface UXO using a magnetic field with a magnetometer is analysed in [48], based on a set of landmine or UXO sensor signatures. It is concluded in these studies that since many target signatures are site-dependent and variable based on the features of UXO, obtaining trustworthy priori training data in advance of designing an algorithm is frequently challenging. Considering this conclusion, the techniques developed in our research employ field-dependent data sets, without requiring a priori training set. The self- and user-selective threshold classification and clustering mechanisms help reveal MF distinctive from the rest of the Area of Interest (AoI) as elaborated in Section III.
The integration of magnetometer sensors with small UAS is carried out by various studies to realize different objectives such as [49] and [50] in increasing the quality of magnetic field by reducing the permanent and induced interference magnetic field generated by the drone. We aim to increase the quality of the magnetic field in our novel drone and sensor integration design as explicated in Section III-B. The effectiveness of drones equipped with magnetometers in detecting buried metallic explosives, in particular, AP and AT landmines, was demonstrated in various studies [51], [52], [53]. We analysed the initiatives of using drone-mounted magnetometer systems in the market. The magnetometer-mounted UAS have been developed to provide an integrated solution to demining operations as demonstrated in Figs. 2, 3, 4. The features of these UAS are summarised in their legends. These systems are yet to provide an ideal compact system that the market demands as elaborated in Section V (Table 6). Millions of buried landmines still need to be found and removed manually, despite significant efforts to identify landmines using automated remote sensing approaches and using these manual techniques, it would take hundreds of years to fully demine all of these mines. It is now critically necessary to develop landmine/UXO/IDE detection and removal systems quickly [3] where their removal is very risky, expensive, and time-consuming [4]. The incorporation of aerial surveying supported by drones and multiple sensor modalities seems to be the most viable option to expedite the demining, specifically, in tough terrains. In this paper, regarding the previous promising studies on magnetometer sensor modalities, we have built a new integrated holistic system to detect landmines/IDE/UXO automatically in large terrains using UAVs. To the best of our knowledge, this research is the first attempt to determine the likely locations of potential landmines/IDE/UXO autonomously, rapidly and safely using a bespoke, lightweight, small and intelligent aerial-based, integrated, and easy-to-use compact drone (quadcopter) system equipped with a magnetometer sensing system and live sensor data telemetry link, which meets most of the market demands as explicated in Section V (Table 6).
Methodology
A. Background
This research is based in The University of Central Lancashire (UCLan)’s Engineering and Innovation Centre, a 35m building bringing together additive manufacturing, software investment of £ 1.3M in 2021 to procure drone equipment to support local businesses and enable new research.5 Many commercially available geophysical ground scanning sensors were procured and bespoke ones were developed. These have been utilised and evaluated over the last few years in helping solve real-world problems intelligently. UCLan has developed many bespoke autonomous small, lightweight, compact quadcopters equipped with sensors for different types of objectives (e.g. for agriculture [59], [60], landmine/UXO/ IDE detection [1], collision avoidance [17], beyond visual line of sight (BVLOS) teleoperation [61], [62]). UCLan has been collaborating with the Cambodian Army and several landmine-cleaning-based NGOs to develop new approaches and improve the pre-developed techniques for detecting and demining landmines. The Aerospace and Sensing Research (ASR) team at UCLan tested drone-mounted magnetometers with Cambodia’s Armed Forces Peacekeeping Division.
The ASR team was previously funded by both the Global Challenges Research Fund (GCRF) in 2018 and the Internal Engineering Research Centre Fund in 2021 in developing landmine/UXO/IDE applications. The performance of particular remote sensing sensor modalities such as GPR, magnetometers, infrared (IR), a Longwave Infrared (LWIR) camera, and a multispectral camera has been evaluated in-field tests. The fusion of data obtained from the integrated GPR and magnetometer sensor modalities mounted on an autonomous UAS (Fig. 5) has already accomplished satisfactory results with very high accuracy rates in finding landmines [1] (Figs. 6), 8). Initial datasets using visionbased remote sensing sensor modalities (i.e. IR, LWIR camera, and multispectral camera) were collected in Croatia in 2018 [63]. Later, the developed sensor-integrated UAS were tested in Cambodia in larger mine-affected areas in cooperation with the Cambodian Army and NGOs to quantify the observed results in difficult scenarios. Two landmine sites (UCLan Hawkins yard and Myerscough site (Figs. 8, 9) were already designed with landmines/ IDE/UXO for scanning by drones in Preston, UK. Recently, UCLAN and Qatar University have established collaboration6 in developing drone-mounted sensor systems to support the landmine/UXO/ IDE humanitarian clearing activities.
B. Design and Robotics Integration of Maggy
We planned to use a small single-board computer (SBC) on Maggy to process the internal management of its parts as well as the sensor components. Arduino and Raspberry Pi are both suitable to our design and development objectives. In this application, the Arduino board was selected to execute simple sensing operations from the sensors where i) it is cheaper than the Raspberry Pi, which helps us to accomplish one of our objectives – a bespoke drone as less expensive as possible and ii) it needs less current than Raspberry Pi does, which is important for us regarding the batteryconstrained Maggy for the extention of flight time. This section consists of two subsections (Sections III-B1, III-B2), i) design and development of the drone – Maggy – with sensor technologies (Figs. 14, 15), and ii) development of the tablet/ smartphone application (Fig. 16) to manage Maggy and process data streaming from Maggy to locate landmines/IDE/UXO.
1) Integration of Maggy with Sensors
The incorporation of the internal software and hardware components with the sensors into the bespoke Maggy system is explained in this section. Fluxgate magnetometer sensors were used to detect MF generated by the metallic parts of landmines, UXO or IDE. Magnetometer sensors should be integrated with UAS appropriately concerning the magnetic interferences relating to onboard electronics as elaborated in [64], [65], and [66] even though the small electronics of Maggy help reduce the interferences significantly. The magnetometers were integrated below a lightweight drone to minimise magnetic interferences, specifically, caused by the UAS (Fig. 17). The properties of the magnetometer sensors shown in Fig. 10 are presented in Table 1. Two fluxgate sensors – magnetometers – are connected to Arduino using the serial port via the Modbus multiple connections as demonstrated in Fig. 11. One of the magnetometers is placed on Maggy to collect MF data via the Z direction and the other is placed to collect via the X direction. The sampling rate was adjusted to 10 Hz in order to reduce the noise (Fig. 17). Sensor data is read as shown in Fig. 12 and programming of sensing is executed using Python as displayed in Fig. 13.
The general features of Maggy considering its drone components are presented in Table 2). The inner design of Maggy is demonstrated in Fig. 14. Each full battery can perform up to 4 min 30 sec at low speed flying (i.e. 1 m/s). An altimeter was incorporated into Maggy to make the flights accurate under 1 meter, enabling reliable terrain-following flight. The “position mode” is the easiest to fly with the centre stick configuration. Maggy uses a distance sensor (i.e. altimeter) for “position hold” below 1 m altitude. In “altitude mode”, Maggy will drift with the wind and is sensitive to control input. The “transmitter timer” is set to 4 min and will start to beep to notify “low battery”. The particular features of Maggy shown in Figs. 14, 15 considering operational objectives are explained in Table 3. By integrating wireless communications with antennas using telemetry radios for remote control, WiFi for real-time data transmission using a 5G Netgear Router and a drone flight controller for precise navigation – we can implement a provision of real-time data which opens up many operational advantages as elaborated in next subsection III-B2. X, Y and Z component directions of the magnetometers are processed as formulated in Eqs. 1, 2, 3) to result in the total magnetic strength/intensity. A Gaussian low-pass filter as well as a highfrequency pass filter are applied to the acquired signals (Fig. 17) to suppress the background noise and accomplish a satisfactory signal-to-noise ratio (SNR) (Figs. 27, 28), which help detect smallscale MF caused by the targeted explosives with metallic objects. The autopilot control system of Maggy was optimised for flight close to the ground, integrating a radar altimeter into the drone to enable terrain following flight at a distance between 50 cm and 1 m above the ground to maximise the sensor performance.
2) Development of the Application
An intelligent tablet/smartphone application was developed using the Xamarin.Net development platform. The Xamarin platform enable us to create an application which can run on both Android- and iOSbased devices. The functionalities of the application are explained in Fig. 16. It was fully integrated with Maggy to i) manage Maggy, ii) process data streaming from Maggy to locate landmines/IDE/UXO, iii) perform detailed survey analysis considering varying MF, and iv) communicate with the landmine/UXO/IDE clearing team for reporting the exact locations of explosives. From a technical standpoint, the application establishes an agreed-upon communication link with Maggy using either a TCP or UDP connection. Preferably, a UDP connection is suggested to be used where each data point read by Maggy needs to be readily displayed on the application without stricter protocols as in a TCP connection. Maggy can be used in an automated manner where planned waypoints can be fed into Maggy using the UgCS system – drone flight planning software. Maggy transmits MF values with related information at each data point on its waypoints to the application. The flight information and MF data are streamed to the application to be processed and monitored in near real-time. The attributes of each data point are explained in Table 4 with an example. The streaming of data was coded using Python and the Python script codes of streaming (Maggy_UART. py) are provided in the supplementary materials for interested readers. The streaming is communicated through 5G Netgear Router’s WiFi connection as mentioned earlier. The application readily processes these values using Eqs. 1, 2, 3 for MF classification and clustering based on the MF threshold chosen by the user as explained in Fig. 16 and shows landmine/ UXO/IDE GPS locations on the local map with abstract information (Figs. 27, 28) as data is streamed from Maggy. The classification of MF values is carried out based on the distribution of the MF values obtained from various landmine/UXO/ IDE devices considering the “no MF” values as exemplified in Section IV-A. Regarding the clustering, values below the threshold value are ignored and clustering is executed based on these values above the selected threshold. These algorithms are employed to classify the MF values as “very high MF” represented by “red”’ colour, “high MF”’ represented by “orange” colour, “low MF” represented by “yellow” colour, and “no MF” represented by “green” colour. This is demonstrated in Section IV, particularly, in Fig. 27. The use of the application with its functionalities is further explained in Section IV-B with real-field implementations.
© 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License.
Originally published in IEEE Access, VOLUME 12, 2024. Republished with authors’ permission.
To be continued in next issue.
(1 votes, average: 5.00 out of 5)
Loading...
Leave your response!