Assessment of accuracy of basic maneuvers performed by UAV

In the paper a field base for testing the accuracy of positioning unmanned aerial vehicles has been presented, as well as a set of basic tests allowing to define the parameters of accuracy in particular mission scenarios

Pawel CWIAKALA AGH University of Science and Technology, Faculty of Mining Surveying and Environmental Engineering, Krakow, Poland

The use of UAVs in various works is becoming increasingly popular. They are commonly used in surveying as well as preparation of photographic and video documentation of various objects [5],[7].

The introduction of shipment services, even including transport of blood or human organs, is being discussed more and more frequently. Growing requirements for ensuring high level of safety in aviation operations, as well as legal regulations, will force the certification of UAV equipment. According to the author, one of the stages of certification of this type of equipment should involve tests allowing to define characteristics determining the correct performance of tasks by the UAV. The common definition of the UAV applied for aerial vehicles used for purposes other than recreational or sporting ones, says that it is an aircraft that does not require crew on board in order to fly, nor is it able to pick up passengers, it is piloted remotely or it can conduct autonomous flights. On the other hand, the definition of an industrial robot is that it is an automatically controlled, programmable, multitasking, manipulative device, with multiple degrees of freedom, mobile or stationary, for heavy industrial applications [8]. Considering the above definitions, UAVs can be classified as mobile industrial robots. This paper analyzes tests presented in ISO 9283:2003 [9] for the possibility of their implementation in measurements of UAV positioning accuracy.

The accuracy of an ndustrial robot is determined by its functional characteristics [11], the most mportant of which include the accuracy and repeatability of reaching a position. Guidelines ßor testing the accuracy of robots are described by the PN-EN ISO 9283:2003 standard ([9], [10]). Different measuring methods can be used for examining the ositioning of such devices ([1], [2], [12], [13], [14]).

Table 1 lists all the tests that should be performed when testing an industrial robot.

This paper discusses works allowing to perform the following tests and to define the performance parameters listed below:

– positioning accuracy, understood as the difference between the commanded position and the average pose attained by the robot, when the commanded pose is attained from the same direction,

– positioning repeatability, i.e. the measure of scatter of deviations between positions attained after n repetitions, with the same commanded position attained from the same direction,

– multidirectional positioning accuracy variation, which is the maximum distance between the average of the attained positions, obtained with the same position commanded n times from three perpendicular directions,

– distance deviation, which is the difference between the commanded distance and the average distance attained,

– distance repeatability, or measure of scatter of deviations of attained distances, where distances attained correspond to the same distance commanded, repeated n times from the same direction.

Description of performed field tests

The research was carried out based on the test base composed of eight points. Their coordinates in the WGS- 84 system were determined using RTN technology with reference to the ASG-EUPOS network. The entire base consisted of two basic components:

– three points which constituted a measuring stand and reference points for measurements using the Leica MS 50 total station, which is a tool allowing to capture large data sets of a moving object in a short time and with high accuracy,

– five points constituting endpoints of the sections tavelled by the UAV, which were positioned on the base in the form of a cross, whose arms were parallel to the directions of the world, each arm was 25.00 m long (from the center of the base).

The placement of individual components of the base ensured optimum operating conditions for the Leica MS50 total station, equipped with the ATR system to track the prism installed on the UAV. The Leica MS 50 was positioned above the UAV start point, on which the mini 360º prism was placed. Figure 1 illustrates the position of measurement base points. Before the commencement of each measurement, a test was performed to observe a fixed target for a minimum of 30 seconds. Thus, it was possible to assess the measurement accuracy of the instrument. Spatial deviations from the average position of the fixed prism did not exceed 15 mm which, at their standard deviation of 10 mm, proves that measurement accuracy of this instrument is satisfactory for testing UAV devices equipped with GNSS navigation receivers.

The test unit was the DJI S900 Spreading Wings Hexacopter equipped with a DJI A2 on-board computer. Thanks to all the on-board systems, the hovering accuracy of the DJI S900 declared by the manufacturer is (in GPS ATTI Mode): ±0.5m (vertical) and ±1.5m (horizontal). Additional parameters of the flight controller are: maximum wind resistance (less than 8 m/s), maximum yaw angular velocity (150°/s), maximum tilt angle (35°) and maximum velocity of ascent/descent (6 m/s) [3], [4].

All the tests were performed so that the center point of the test base was reached from four independent directions related to the directions of the world. The UAV was programmed to perform the mission so as to reach the commanded center pose at least 25 times. The number of tests was determined by the time of flight on one power cell. The test was performed at the height of 30 meters, with a wind speed not exceeding 5 m/s, so that the measurement results were not affected by turbulence associated with trees and other infrastructure around the measurement base.

Research results

The measurements resulted in coordinates of the points defining the position of the UAV in space. During the measurements, frequency of recording coordinates of the points was 5.4 Hz, which is less than the value of 20 Hz declared by the manufacturer of the Leica MS 50. It is, however, sufficient for further calculations and analyses. The data were calculated to define the characteristics listed in previous section. The calculations were carried out so that description of the results was related to the direction of approach to the center point, from north (N), south (S), east (E) and west (W), respectively.

Positioning accuracy

Table 2 demonstrates results of the UAV positioning accuracy test. The columns contain parameters for the respective axes of the coordinate system: the north and the east ones, as well as for the height. It is important to note that the standard requires only spatial values for the center of gravity of the position attained by the robot from the same direction, which is the fourth column.

It should be noted that in all tests, differences in the north direction are much higher compared to the east direction. Considering the accuracy of A2 on-board computers guaranteed by the manufacturer, the results prove that positioning of the device with the commanded accuracy (±1,5m) is not ensured. In addition, it should be emphasized that in the case of height positioning, the difference is significantly higher than values declared by the manufacturer (±0.5m). Figure 2 illustrates values of spatial differences between each of the center pose attained during the tests, in relation to the one predetermined for each direction of approach.

Fig. 3 illustrates a graph of values of the differences between each center pose attained during the test, relative to the one predetermined for each direction of approach, broken into components of the coordinates. The Figure demonstrates that the center point in subsequent measurement tests was attained with the repeatability encumbered with average shift resulting from the GNSS receiver positioning accuracy.

Positioning repeatability and multidirectional positioning accuracy variation

The results of positioning repeatability are summarized in Table 3. It should be noted that this performance parameter defines accuracy of implementation of a commanded position, having eliminated the systematic factor related to the GNSS receiver solution inaccuracy. Taking its values into consideration, it can be stated that direction of approach to the center point does not affect the positioning repeatability of the device.

Multidirectional positioning accuracy variation is represented by one parameter, which in this case is 1.08 m. The value of this parameter defines the maximum difference between the mean values of the pose attained from each direction.

Distance deviation and repeatability

Distance deviation parameters for each direction of approach to the center point are summarized in Table 4. It is worth noting that, in all directions, the length of the section covered was shorter than the predetermined section. In the case of classic industrial robots, this indicates a scale error. Table 5, which summarizes the covered distance repeatability, demonstrates that this parameter exceeds 1 meter. Therefore, it can be assumed that the previously discussed values of distance deviation can be regarded as negligible. In addition, Figure 4 illustrates differences of all UAV-covered sections, depending on the approach direction.

Discussion of results and summary

The performed tests and calculations allow to draw a number of conclusions. First of all, attention should be paid to practical observations resulting from field research. The use of the Leica MS 50 total station yields satisfactory results. In addition, it is important to note that wind has a significant effect on the trajectory of an implemented mission. Therefore, it is best to test flying equipment in at least two scenarios: in windless weather and in conditions which are close to boundary ones for a specific on-board computer. It is worth emphasizing that the proposed set of tests can be considered as the first element in testing performance of on-board computers.

According to the author, research should be conducted, which would be aimed at building a set of tests allowing for a full, independent assessment of the UAV’s performance in terms of accomplishment of missions in space. Considering dynamic development of numerical image analysis methods, it is possible to attempt to determine the UAV’s position during a flight, based on terrestrial imagery [6]. The UAV positioning using GNSS PPK technologies may also prove helpful. Moving on to analysis of the results, it is necessary to start with positioning accuracy. The parameter provided in specification of the DJI A2 on-board computer is unclear in interpretation. Comparing it with positioning accuracy seems to be the best reasoning.

The measurement results and obtained characteristics are not consistent with the data provided by the manufacturer. Positioning repeatability and multidirectional positioning accuracy variation reach values smaller than those declared by the manufacturer.

This result is most likely related to the systematic shift of the UAV coordinates determined using GNSS technology. When calculating these parameters, such systematic errors are eliminated. This thesis is confirmed by the results for distance deviation and repeatability. To conclude, it should be stated that creating a suitable and secure field base, as well as a set of tests to define the UAV’s behavior during a flight, will allow to determine boundary conditions for applicability of this type of equipment in variety of mission scenarios.

Acknowledgments

This paper was written with the support of the AGH-UST statutory means, number 11.11.150.005.

References

[1] Aoyagi S., Kohama A., Nakata Y., Hayano Y. Improvement of robot accuracy by calibrating kinematic model using a laser tracking systemcompensation of non-geometric errors using neural networks and selection of optimal measuring points using genetic algorithm. 18–22 Oct. 2010, pp. 5660–5665.

[2] Ahmad R. A., Azmi H., Syamimi S., Mohd Z. N., Shahrum A. The Repeatability Analysis of Industrial Robot under Loaded Conditions and Various Distances. International Journal of Mathematical Models and Methods in Applied Sciences, 2008.

[3] DJI Spreading Wings S900. User Manual v.1.4, 2016.

[4] DJI A2 Flight Control System. User Manual v.1.24, 2016.

[5] Fahlstrom P.G., Gleason T.J.: Introduction to UAV Systems. John Wiley & Sons, 2012.

[6] Garcia Carrillo L.R., Rondon E., Sanchez A., Dzul A., Lozano R.: Stabilization and Trajectory Tracking of a Quad-Rotor Using Vision. Journal of Intelligent & Robotic Systems, vol. 61, pp 103-118, 2011.

[7] Gontarz A., Kosielinski S. (Ed.): Polish drone market, 2015. Book of supply and demand. Mikromakro Institute Foundation. Warsaw, 2015.

[8] ISO 8373:2012 Manipulating industrial robots – Vocabulary.

[9] ISO 9283:2003 Manipulating industrial robots. Performance criteria and related test methods.

[10] ISO 9787:2013 Robots and robotic devices – Coordinate systems and motion nomenclatures.

[11] ISO 9946:1999 Manipulating industrial robots – presentation of characteristics.

[12] Kluz, R. and Trzepiecinski, T. The repeatability positioning analysis of the industrial robot arm. Assembly. 2014, Vol. 34, 3, pp. 285-295.

[13] Renders J. M. et al. Kinematic Calibration and Geometrical Parameter Identification for Robots. IEEE Transactions on Robotics and Automation, Vol. RA- 7, No. 6, 1991, pp. 721-731.

[14] Zurek J., Wisniewski M. Methodology and conditions of testing accuracy and positioning repeatability of industrial robot. FANUC M-16iB. Technology and Assembly Automation. 2013, Vol. 1, pp. 31-34.

The paper was presented at INGEO 2017 – 7th International Conference on Engineering and Surveying, Portugal, October 18-20, 2017.