Drone Technologies: A Tertiary Systematic Literature Review on a Decade of Improvements

Unmanned aerial vehicles (UAVs) have emerged as versatile tools with significant potential in various fields, including, but not limited to civil engineering, ecology, networking and precision agriculture. Systematic literature reviews (SLRs) play a crucial role in assessing the quality of research methods and approaches, aiding researchers and practitioners in selecting and optimizing their projects. However, the quality assessment of UAV-related SLRs and the aggregation of UAV technologies across research fields remain limited. This study aims to address these gaps by conducting a tertiary literature review (TLR) that assesses the quality of SLRs, aggregates data across research fields, and provides guidelines for researchers and practitioners in the UAV community. Based on a review of 73 SLRs, it is evident that the quality of UAV-related SLRs is generally low, with a lack of quality assessment and inadequate reporting of detailed information on primary studies. Consequently, this study presents reporting items and example quality assessment ratings sourced from the SLRs to enhance the transparency and comparability of future UAV-related research. Additionally, it highlights common limitations faced by UAV applications, such as regulatory, technical, social, and research-related challenges, which require attention for progress in the field. Overall, this study aims to enhance the quality and knowledge sharing in the UAV research community by providing insights into the current state of UAV-related SLRs and offering practical guidance for researchers and practitioners through the provision of data-extraction templates and quality control questions for future UAV-related reviews and primary research.


I. INTRODUCTION
The past decade has seen the arrival and proliferation of commercial drones.In these first years, drones were experimentally tested for various tasks such as agricultural crop detection, search-and-rescue operations, and medical goods transportation.However, decreasing costs and simpler operations made Unmanned Aerial Vehicles (UAVs) an increasingly common tool for data acquisition, transport, and The associate editor coordinating the review of this manuscript and approving it for publication was Jiankang Zhang .real-time operations across industry, the public sector, and more academic niches.
This widespread adoption of UAVs has created a great diversity in UAV users.Each of these practitioners has created their own solution to apply UAV technology to their particular use case.For example, in a review on UAVs for supporting algal bloom research, [1] identified 10 unique sensors on 10 different flying platforms, used in four different ecosystems, with the use of seven different spectral indices.Whereas in a review on UAVs for parcel and passenger transportation [2], six different problems facing UAVs for transportation and three different solutions were presented.Furthermore, in another review on UAVs for forest firerelated usage [3], eight different UAV characteristics for fire extinguishing tasks, ranging from UAV group size to fire-extinguishing tools and collision avoidance algorithms, covering 51 articles were found.These examples demonstrate that for each use case, depending on the UAV model and payload, unique solutions are created [1].
These solutions require a high number of resources to develop, raising the barrier to entry [4].For example, UAVs can be more discrete than humans for monitoring vulnerable species [5], although complex sensors and data processing pipelines need to be developed for the best results [6].In addition, developing countries with high forest cover have very little published UAV research [7], even though these forests have the highest need for careful monitoring programs.Another example is in healthcare, where UAV transport of medical goods is often faster than the standard response, however, a medical payload also requires a clinical and sterile container, secured onto the UAV [8], which increases research and development costs.Lowering the barrier to entry can therefore be supported by identifying which technologies, practices and methods have been more prominent in research.
An important method in sharing these technologies, practices, and methods is a Systematic Literature Review [9] (SLR).SLRs originated from medical sciences and were adapted for Software Engineering by [10].This adaptation presents a systematic methodology for searching articles, extracting information, and rating the study using quality assessment criteria.
A review of papers using the SLR method was published by the same group three years later [9].Their findings indicate that the literature reviews are not in accordance with any method, many do not perform a quality assessment, and only a few of their included articles present guidelines for practitioners, which is one of the primary benefits of a literature review.This 'review of reviews' is often called a tertiary literature review (TLR).Different from an SLR, a TLR aggregates information only from secondary studies [9].A TLR is performed to consolidate information across fields; identify research topics; aggregate scattered data; find limitations and assess the quality of SLRs with the DARE rating scheme [11].In [12] for example, the TLR method from [9] was adapted to Machine Learning in Software Engineering.The impact of Machine Learning on the field was assessed and a classification scheme for categorizing applications was presented.Additionally, the quality assessment of review articles indicates an average rating of 3 out of 4, using the DARE rating scheme [11] (more on this scheme will be presented in Section III-C).
To the best of our knowledge, only a single TLR has been published within the UAV field.The TLR from [13] covers UAV use in civil engineering.They identified concerns for UAV deployment and future directions for using UAVs in civil engineering.More problematic however is that the results of their quality assessment on UAV reviews show low results: on average the reviews got a quality rating of 0.3 out of 3 [13].Only 3 quality criteria were measured from the DARE rating scheme in the TLR.The authors indicate that the low score is mainly from a shortage of quality assessment being performed in their included SLRs.Additionally, the TLR did not aggregate data on UAVs across research fields.
SLRs are a valuable tool to assess the quality of methods and approaches, helping researchers and practitioners select, and optimize their projects [10].However, when the quality assessment is not performed, or an aggregation of data is not executed, researchers and practitioners miss out on key information to reduce the barrier to entry for using UAV technology [4].To support research and practitioners of UAV technology, an understanding and improvement of SLR quality is needed in addition to a broad aggregation of UAV technologies used across research fields.The aim of this study is therefore threefold.We aim to: In this study.a tertiary literature review is conducted.This TLR assesses the quality of SLRs, aggregates data across research fields, and can provide broad guidelines to be applicable across the widespread UAV research community.This aim is focused on providing items for practitioners and researchers, which is reflected in the main research question: How can future UAV studies be conducted to improve the quality for optimizing information sharing for practitioners and researchers?
This question consists of the following sub-questions: • RQ1.What is the quality of systematic literature reviews conducted in the UAV domain?
• RQ2.What are the popular UAV platforms, payloads, and software in the UAV domain?
• RQ3.Which limitations for using UAVs are identified in the SLRs?
Our TLR on 73 SLRs reveals that the quality of SLRs on UAVs is not high: SLRs are not performing a quality assessment step.Additionally, SLRs do not report detailed information on their primary studies, making comparative analysis difficult.Therefore, various reporting items and example quality assessment ratings, sourced from the SLRs, are presented in the supplement of this article for primary and secondary research.Finally, many UAV applications are facing similar limitations, solving these regulatory, technical, social, and research-related limitations will be a fruitful endeavor in the coming years.
The main contributions of this review are as follows: 1. providing a database of UAV reviews and UAV technologies, 2. assessing the quality of reviews, and 3. publishing data-extraction templates and quality control questions for future UAV-related reviews and UAV primary research.

VOLUME 12, 2024
The article is structured as follows: Section II covers related work, Section III presents the TLR methodology, Section IV presents the results, Section V presents the discussion, Section VI identifies threats to the validity of this study and Section VII presents the conclusion.

II. BACKGROUND AND RELATED WORK
The technology for UAVs has a background in military usage.The design of large aircraft such as the NASA Ikhana and Sierra were created with high-altitude surveillance or even tactical operations in mind [14].Over time, microchips became faster and smaller, and battery life and power output increased, making smaller and more affordable UAVs available to a wider audience.These smaller aircraft were the main subject of early work in using UAVs for various non-military use cases which at the time were classified as small, mini, micro, and nano-sized UAVs [14].The overview of different UAV platforms and the operational range provided is presented as follows: large UAVs have an operating range of approximately 500 km and can fly for up to 2 days; medium UAVs have a similar operating range but a shorter flight time of around 10 hours; small UAVs have an operating range of fewer than 10 km and a flight time of fewer than 2 hours; mini UAVs can fly with up to 5 kg and typically require hand-launching; micro and nano UAVs have even more limited operating ranges, and have very short flight times of less than 1 hour.
In the article from [14], the authors argue that these lightweight (small, mini, micro, and nano) UAVs will revolutionize spatial ecology.The birds-eye perspective, in combination with the high-resolution sensors and usercontrolled revisit periods, is posed to fill the gap of data required by spatial ecology, at an affordable cost to many researchers.They present three usages of UAVs for spatial ecology monitoring: 1. population ecology through thermal sensing; 2. vegetation dynamics through multi and hyperspectral sensors; and 3. ecosystem processes through miniature temperature, pressure, and particle sensors.
A similarly influential work by [15] reviews the use of UAVs for photogrammetry and remote sensing.Their classification extends previous works by separating fixed-wing and multi-rotor aircraft within the lightweight categories.They provide an overview of various technologies that make up a UAV system.The overview presented by the authors starts with the flying platform itself, with the various classifications and usages, a ground-control station, communication between the various parts of the UAV system, and mission planning.Regulatory bodies and regulations are covered for various regions of the world.These mainly cover working groups, preliminary regulations, and older regulations affecting these new small systems, whilst arguing for the renewal of many of these regulations.Various payloads for the UAV are presented, in the form of autopilots and navigation systems, orientation systems, and sensing payloads.Critically, many examples are given of the available technologies.In the processing section, an overview of image orientation and camera calibration methods is presented and the state-of-the-art for surface reconstruction is shown.They identify that the high spatial resolution that these new UAV platforms create poses a new problem to existing photogrammetry methods: the requirement of having extremely accurate positioning information for accurate image alignment and surface reconstruction methods.They conclude that there is a high variance between UAV studies, therefore the extrapolation of results to other UAV studies has to be made with caution.
In the years since these initial forays, many studies, surveys, and reviews have been performed on the use of UAVs for various applications across academia.
The aforementioned study from [13] is the only tertiary study we identified, which covers 33 secondary reviews on UAVs for civil engineering and construction.Their tertiary study identified three main concerns with UAV deployment: restrictive regulations, system reliability, and factors affecting UAV use.Future directions for UAV studies were directed to the development of sensors, algorithms, navigation, and capabilities.Additionally, they propose more studies into the feasibility, human-UAV interaction, and effectiveness of specific UAV solutions.Finally, the study found that the quality of UAV research reviews is quite low: on average the reviews got a quality score of 0.3 out of 3 [13].
A similar topic of UAV technology and best practices is also covered in the study by [4].Their focus is on presenting onboard UAV sensors and their best practices for calibration, improving mapping quality, emerging acquisition scenarios, trends in UAV data processing, and future challenges for UAV technology.Each of these topics is covered by providing examples from research or industry.Additionally, in the case of larger research topics, the reader is directed toward review articles.The presented best practices are clear and actionable, usable for many starting and experienced practitioners.Examples include how ground control points should be placed for the best results for photogrammetry, tips on battery life optimization, and how to calibrate a multispectral sensor.Finally, the authors argue that more cross-pollination in UAV research is required, by starting from the perspective that UAV applications are scattered, and that the technology depends heavily on progress in other research fields.Such as increased battery efficiency, computer vision research, and miniaturization of processing power [4].
To complement these prior studies, we conduct a systematic tertiary literature review that will analyze the quality of prior review articles, systematically aggregate data from these reviews and identify a set of common limitations across UAV research.Our study is different from prior studies in that we systematically reflect on the SLR method itself, to provide guidelines for future SLRs and the field as a whole.

III. METHODOLOGY
The Systematic Literature Review method from [10] was adapted for tertiary studies in [9].These guidelines present a literature review according to three phases: planning, conducting, and reporting.During the planning phase, a search protocol is developed and reviewed by all authors.This protocol consists of a search process, quality assessment using Database of Abstracts for Reviews of Effects (DARE) criteria, and data extraction.The following section presents the protocol followed and the adaptation to UAV applications.

A. THE SEARCH PROCESS
Three databases and one indexing system were searched for articles.The searched databases were: IEEE Xplore, ACM, and Clarivate Web Of Science.The indexing system was Elsevier Scopus.These were searched between August 9th and September 9th.All the databases had the following query for all fields (title, keywords, abstract): ''UAV'' AND (''review of studies'' OR ''structured review'' OR ''systematic review'' OR ''literature review'' OR ''literature analysis'' OR ''literature survey'') .The terms could have been extended by using RPAS (remotely piloted aerial system), UAS (unmanned aerial system), and drone.The variations for 'review' were directly adapted from [9].
Furthermore, to acquire all relevant information based on the recent usage of UAVs, the results were limited to 2012 onward.These queries resulted in a total of 608 studies.No backward or forward snowballing was executed, as the three databases and indexing system already showed a large overlap in results, and the search was deemed exhaustive.

B. STUDY SELECTION
The 608 studies were included following a process adapted from the Preferred Reporting Items for Systematic Reviews (PRISMA) statement [16].The statement explicitly calls for an identification, screening,and inclusion stage, with specific definitions for each step presented in figure 1.
The inclusion criteria were the following: • Reviews on UAV usage and UAV technologies across all domains were included.The exclusion criteria were the following: • Non-secondary studies were excluded.
• Studies where UAV was not an abbreviation for Unmanned Aerial Vehicles were excluded.
• Articles not written in English were excluded.
• Based on a 'defined search process' were excluded (not reporting search queries and database).
• Inaccessible papers and reports were excluded.
The most important exclusion step for selecting Systematic Literature Reviews was whether the review is 'based on a defined search process', directly taken from [9].This means that the exact search queries and the searched databases were clearly presented in the article in the form of text or figures.These inclusions led to the final inclusion of 73 systematic literature review articles.These SLRs cover a total of 8536 primary sources, however, overlap will exist between the included primary sources.

C. QUALITY ASSESSMENT
Each review was manually evaluated according to four criteria presented in [9], presented in table 1.These criteria are adopted from DARE [11].This is a strict review assessment scheme, initially developed for medical literature reviews, but has seen widespread use across academic disciplines [17].This quality check was executed by the first author.These questions (QA1, QA2, QA3, QA4) were scored (Yes (Y) = 1, Partly (P) = 0.5, No (N) = 0) according to the scheme in table 1.

D. DATA EXTRACTION PROCESS
The data extraction process is presented in figure 2. Quality assessment in figure 2 are the quality assessment ratings as presented in section III-C above.Metadata in figure 2 consisted of the following items: • Quality assessment score • Whether it included grey literature sources • Search year range (start year and end year) • Research category (based on the subject matter) • Whether it contained detailed data extraction on hardware or software Aggregated findings on popular UAV technologies were subdivided into hardware and software.Naturally, only the SLRs that performed a detailed data extraction provided information for this extraction (n=21).Additionally, TABLE 1. Quality Assessment rating scheme.Adapted from [9] and [11].
hardware was split into platform and payload tables.Where the platform is the flying platform, and the payload is the sensor or other mounted component on the platform.This is the result of starting the extraction based on the detailed extraction from [18] and adapting it when other SLRs reported something new, which is a recommended procedure [9], [10].Limitations were identified by screening the SLRs with a QA score over three (n=16) and searching for limitations of UAV use according to the SLR discussion sections.The reader is encouraged to explore the complete data extraction, including comparative tables, found in Appendix A. Supplement I. data extraction.
Below, we explain additional details relating to how the data extraction was done for each research question.

RQ1. What is the quality of systematic literature reviews conducted in the UAV domain?
To elaborate on the SLRs that performed a Quality Assessment, the details on the specific method of Quality Assessment were extracted from these SLRs (n=3).

RQ2. What are the popular UAV platforms, sensors and software in the UAV domain?
The data extraction on UAV technologies as presented in figure 2 had some minor adaptations to be made.For example, some SLRs extracted 'sensor type' and others 'payload model', however, the content is similar.These were all adapted into the same field, called 'payload type'.More importantly, however, a data-cleaning step was included to make variables group-able and comparable.For example payload naming conventions: different spellings for the MicaSense MCA lite Multispectral sensor would all be renamed to: 'MicaSense MCA-Lite'.This became more problematic when an SLR overlooked critical details.For example, a 'DJI Phantom 3' itself does not exist, but a 'DJI Phantom 3 Professional' does exist.Such cases were left as is.An additional grey literature search was performed to identify the key details of each sensor, software and payload.The resulting data extraction tables for this research question are presented as supplementary materials I and II (supplement I. data extraction: UAVDataExtraction[.xlsx/.csv],supplement II.variables explained: UAVDataExtractionSupportingDocument.pdf).
RQ3. Which limitations for using UAVs are identified in the SLRs?The limitations to using UAVs were identified by searching for limitations presented in the discussion section of the SLRs with a quality score over three (n=16).This additional exclusion rule is to limit the total number of articles on which to perform this resource-intensive task and prioritize the findings from higher-quality SLRs, which is promoted as a method by [19].The limitations were categorized under regulatory, technical, social, and research limitations, as these four categories covered all of the recurring limitations.Based on the extracted limitations, as presented in figure 2, it became clear that future research could be supported with predefined extraction tables.These extraction tables were developed using the aggregated extraction on hardware and software presented for RQ2.The aggregation was initially based on the extraction in [18] and continually adapted to fit the extractions performed in the other SLRs.This results in extraction tables that cover most of the variables that UAV SLRs have reported upon UAV model type, analytical models used, the accuracy of the analytical model, sensor settings, etc.

A. OVERVIEW OF DATA EXTRACTION
In this section, the results from the data extraction are presented.This covers metadata of the included SLRs and their topic categorizations.
The number of published review articles steadily increases over time, as seen in figure 3.Although the search query ranged from 2012 to August of 2022, systematic literature reviews only started to be published from 2016 onward [18], [20].Even though the search stopped in August of 2022, the year 2022 already contains the highest number of published articles.This increase is in accordance with the review articles, which indicate a steep incline of newly published work [21], [22].This increase was also expected in the study of [15] as it shows the increasing popularity of the field.
For topics (see section III-D) with more than 1 included review, the growth of individual fields is shown in figure 4. Forestry and Precision Agriculture have seen the biggest increase in systematic literature reviews since 2020.Furthermore, the construction field is also a significant newcomer to publishing SLRs on UAV usage.The rapid increase seems to indicate large growth in future systematic literature reviews in the coming years.
Figure 11 shows that UAV research is active in a wide variety of research fields, ranging from noise pollution from UAVs to forestry, agriculture, traffic monitoring, and networking.Furthermore, figure 11 shows that search ranges are often left as default (not reporting start and end-date) and when they are reported, researchers often set the start date around 2010.This results in some SLRs including UAV research which might be irrelevant to modern usages of UAVs.Additionally, the number of included articles varies highly; some chose to include 3 articles, and others included over 300 articles.For such high inclusion rates, a question can be raised whether the authors can provide quality analysis of the primary sources.the bold text in figure 11 indicates that the review has a quality score over 3, for topics that have more articles, such as Precision Agriculture and Forestry, there are relatively few high-quality reviews.However, high and low-quality SLRs are present across various topics, especially low-quality SLRs seem to be the norm rather than an exception.Finally, the italic text in figure 11 indicates whether systematic data extraction tables are presented in the review, as supplementary data, appendices, or in-text.Only 27% of included articles provide this clear overview of selected studies and/or preferred processing methods.

B. RQ1. WHAT IS THE QUALITY OF SYSTEMATIC LITERATURE REVIEWS CONDUCTED IN THE UAV DOMAIN?
The histogram in figure 5 shows that even though the average quality of the included reviews is 2.13, there is a high peak on the lower side of the quality scores.Not a single study acquired the highest rating of 4.
The average results from the quality assessment are shown over time in figure 6.A slight growth in quality is observed over time.However, the average score of 2.13 (out of 4) is not particularly high.
The lowest scoring quality criteria was QA3 with an average of 0.17 out of 1 (yellow in figure 6. QA3 asked whether the review performed a quality control check on their primary studies, which often is not executed in UAV systematic reviews.Additionally, there were articles that said to have performed a quality assessment but did not report the method of this assessment in a clear manner.For instance, the work of [3] presents four quality assessment questions, but no rating scheme for these questions, nor the results of the quality assessment.
The three articles that reported on a full quality assessment, were [23], [24], [25].In table 2, the quality assessment types of these three articles are described.In [23], most of their included studies score a 'good', which is the middle score, with very few 'average' or 'state-of-the-art' ratings.Reference [24] only reported on a single quality criterion, although they rated their primary research on four criteria.This single criterion is reported as a number between zero and six, which means little without the context, which is not given.Also missing is an overall score on the quality of their primary research.The 20 quality questions and the reporting on the quality assessment performed in [25] is the most useful, presenting an average score of 64% out of 100%.Furthermore, they indicate where primary research is lacking; reporting on soil types, cultivar type, weather conditions, number of taken images, time of day, velocity, and flight duration .They also separated their results according to remote-sensing versus plant-protection-based studies, these two fields indicate similar quality results, and show high overlap in the above-mentioned missing values.
By examining the other low QA scores in this study, QA2 also does not show particularly high ratings either (green in figure 6).This criterion covers whether it is likely that the review has covered all relevant literature.This is measured by looking at the number and diversity of databases that have been searched.Many only include two or three databases (n=30), whilst some searched in only a single database or did not report their used databases at all (n=21).

C. RQ2. WHAT ARE THE POPULAR UAV PLATFORMS, PAYLOADS, AND SOFTWARE IN THE UAV DOMAIN?
Out of the 21 SLRs that performed a detailed data extraction (italic in figure 11), 14 provided detailed information on hardware.For hardware, this resulted in 1208 primary studies.From these studies, different levels of detail were reported, which varies from study to study.Table 3 shows that more studies report on a low level of detail: 'quadcopter' versus a high level of detail: 'DJI Phantom 4 Pro', for payload extraction this difference is even larger.

1) UAV PLATFORMS
In figure 7, the growth of UAV research is clearly visible, as the image is based on primary research.Articles after 2020 have been excluded from this image, as only a few SLRs included articles after 2020.The colour indicates which UAV type was used in the primary research.The availability of multi-rotor UAVs has been the main cause of growth in UAV research since 2012 (pink/purple in figure 7).Most of the used multi-rotors have a quadcopter design.The fixed-wing design has also been used in a significant portion of the published work.Helicopters have almost disappeared completely, although they were sometimes used before 2016.Some prior SLRs may have included balloons as a UAV.However, in recent years, the balloon is not seen as a UAV anymore due to the growth and dominance of the multi-rotor.Not reported in the studies is the hybrid-wing, sometimes called VTOL fixed-wing (Vertical Take Off and Landing).This UAV design contains wings for long-distance flight, and capabilities for hover and horizontal take-off and landing similar to a quadrotor [26].It could be that this design is too recent or experimental for inclusion in the studies.
DJI is by far the most used brand in primary studies (figure 8).No doubt that the advanced autopilot, reliable operation, stabilized cameras, and their ready-to-fly ecosystem influence the popularity greatly.The 'Others' category is also quite large, indicating a high diversity in UAV platforms being used.Furthermore, senseFly is also popular.The senseFly company exclusively develops fixed-wing type UAVs.This type is a smaller section of all used UAV platforms (see figure 7).The focus from senseFly on long flight times and precision mapping no doubt has an influence on the popularity in academia.Other popular brands shown in figure 8, such as HiSystems, 3DR,and AscTec have all gone out of business or merged into larger companies, such as Intel or Leica [27].However, using custom UAVs is also a popular alternative to the off-the-shelve options ('Custom' in figure 8).Perhaps the open hardware and software history of UAVs [14], [15] plays a role in the ongoing popularity of the Do-It-Yourself UAV.
There is a dependence on only a few manufacturers for supplying the flying platforms that are used to perform research.The further implication of this dependence does vary in scale but deserves further attention.

2) UAV PAYLOADS
The most common sensor types over time are shown in figure 9, the digital camera is the most popular.Multispectral and hyperspectral cameras are significantly less used but have seen regular use since the earlier beginnings.Thermal and LiDAR sensors see the least use, although in 2019 and 2020 LiDAR has become more prevalent, perhaps indicating an increase in popularity.
The digital camera is often good enough for many research fields where photogrammetry and/or 3D information suffices to provide insight, such as structural engineering, waste management, and traffic management [28], [29], [30].Also, the simplicity of interpretation and use could explain the popularity.Additionally, the digital camera is also often built into many UAV platforms, such as the DJI Mavic and Phantom line-ups.However, there is a high diversity in sensor use, apart from the most popular ones shown here, mainly used for mapping.The SLRs reported 46 different sensor types.This indicates high flexibility to the UAV, which is creatively used in primary research: there are studies adding particle sensors, gas sensors, microphones, networking equipment, and agricultural sprayers to UAVs.
The popularity of brands for UAV payloads (figure 10) is less skewed to a single manufacturer than the platforms are.Digital camera manufacturers Canon, Sony, Panasonic, and GoPro all have their cameras used in UAVs.Some primary research has modified these cameras to a wider range of light, by replacing the internal infrared filter, making them more akin to a multispectral sensor.Multispectral sensor manufacturers Tetracam and MicaSense also see a significant share of the popularity.DJI is also popular, likely due to the integrated solutions they provide.FLIR focuses on Thermal sensors, Headwall on the hyperspectral side of sensing, and Vaisala on gas and particle sensing equipment.A manufacturer often focuses on a specific sensing niche: hyperspectral sensing for Headwall, Thermal for FLIR and Infratec, Multispectral for Tetracam, etc.The adoption of their sensors within UAV usage is often only a part of their businesses.The outlier here is DJI, which develops a wider range of sensor types specifically for its UAV platforms.The payload brand popularity plot (figure 10) only shows the 188 primary articles on which data was extracted on brand and model, so might be less representative of popularity in the real world.

3) UAV SOFTWARE
Out of the 21 SLRs performing detailed data extraction, 11 SLRs performed a detailed software data extraction.A study using UAVs is built on various software: a mission planner app for the planning stage; flight control software and perhaps experimental obstacle avoidance algorithms in the flight stage; and the analysis stage might make use of Deep Learning, photogrammetry, or a simple linear regression.Therefore, an important distinction must be made in UAV software between software packages (e.g.QGIS 3.11, Agisoft Metashape or PyTorch) and data processing approaches (e.g.spatial overlap and joins, Structure from Motion or Yolov5 object detection).The exact software package that was used in the primary source is often not reported: only three out of the 11 SLRs with a software extraction [1], [31], [32] reported on the software packages.Additionally, confusion exists: [22] provided a software package for a data processing approach.From these four SLRs, 32 different software packages, for 10 different usages have been reported, the list in table 4 is by no means a comprehensive list of all available packages, but a broad categorization based on these four SLRs.Additionally, these four SLRs all report on some form of UAV usage for image acquisition, creating a bias toward post-processing software packages, and missing real-time packages such as ROS and its library of real-time robotics software [33].
Additionally, table 4 does not report the data processing approaches (such as quantitative inversion modeling , Deep Learning or Structure from Motion).Nine out of 11 SLRs reported on data processing approaches.Only [31] reported on both the data processing approach and the software packages used in the primary source.However, all of the nine SLRs that systematically extracted the approach have only reported on a single model in the approach.Three SLRs [22], [34], [35] also included the accuracy score of the corresponding model, such as r 2 for a quantitative inversion model [34].Problematically, a processing approach is more than just a single model and its accuracy score: a processing approach can be performed in multiple software packages, and/or combine multiple approaches and packages.For example, a primary study included in [35], shows a biomass estimation approach for forestry [36].This primary study reports on five steps: 1. a UAV was flown to acquire a set of images of a forest (not mentioned which flight planning approach or package has been used), 2. these images were processed into a 3D model (using Bentley ContextCapture), 3. a canopy height model (CHM) was calculated using Terrasolid, 4. the CHM was imported in eCognition to perform the tree-segmentation algorithm, and finally 5. a linear regression model was fitted to the data to estimate the biomass for each tree in the dataset (not mentioned which software package).The example shows how complex a single approach can be.Systematically extracting a complete approach is therefore quite difficult, which is perhaps the reason why none of the included SLRs attempted it.

D. RQ3. WHICH LIMITATIONS FOR USING UAVS ARE IDENTIFIED IN THE SLRS?
Different research fields have adopted UAV technology as part of their toolkit.These adoptions range from fields that are already deploying UAVs daily [29], to others that are just getting started [8].Cross-pollination between these fields is seen as an important step in improving and adapting UAV research as a whole [4].This section presents a shared set of limitations for deploying and researching UAVs in varying research fields.Four recurring limitations were identified from the articles with a quality score over 3 (bold in figure 11, n=16): regulatory, technical, social, and research limitations (see table 5).
Regulations are lagging behind the technology [30], [37], [43], resulting in vague criteria, such as noise levels [38].Or regulations not being adaptive enough for disaster response activities [39].Additionally, companies can take advantage by implementing unregulated technologies, with the prospect of a (temporary) monopoly [44], resulting in unequal access to this technology and/or its benefits [25].Finally, regulations can be too limiting, many regulatory frameworks do not allow for beyond-visual-line-ofsight (BVLOS) flights [38], which are required for future autonomous applications.
Technical limitations differ for various fields: each field has its own demands.Fields using UAVs for sensing identify that the top-down imagery obfuscates details in plants [31], requiring more advanced flight paths or levels of autonomy [24].Additional image processing limitations exist in low-quality of photogrammetry [29].Correctly timing flights accounting for solar intensity, cloud cover and operational times is also a problem identified in research [25].For other fields, the speed of communication between UAV operators and taking action can be too slow [3].The need to acquire data and also process this data slows down the realtime decision-making, required in emergency response and healthcare [3], [42].However, almost all fields identify the short battery life [8], [25], [40] as a problem.Larger areas that need to be covered require multiple flights, during which the weather and environment could change.Additionally, a high level of autonomy for UAVs is required for many advanced UAV usages, such as parcel delivery [24], [42].
Social factors are similar across various research fields, and include A negative public perception of UAVs due to propeller noise [38] and the military history [8].UAV platforms are inherently dangerous devices, with multiple propellers being powered by powerful motors, and when one cuts out, the whole system falls to the ground.This means that the safety of operation towards humans, animals, buildings, and other vehicles always needs to be taken into account, limiting adoption when this safety cannot be ensured [3], [24], [30].
Research limitations are also prominent, the SLRs are critical of the primary sources [25], [29].The primary research covered in [31] on Deep Learning (DL) application to UAV imagery in forests uncovered that only a small number of primary studies used Deep Learning to a high standard.This high standard entailed the complexity of the Deep Learning task: single tree classification in a monoculture forest is not very complex, and segmenting treetops in a mountainous deciduous forest is a more complex task.Most studies only solved the simple task [31].Another observation that was made concerned the testing-training split.A large number of their primary studies made at least one mistake with testing-training splitting.One example is splitting the data late in the analysis, resulting in a model that already has seen the test examples.Another example is splitting the data by randomly selecting pixels, this should be done by geographically splitting the data based on regions, preferably a different flight or area entirely [31].Furthermore, studies lack a validation step in their research [3], [24], [41].This validation can be a field test, comparing versus other state-of-the-art, or groundtruth validation.Similarly, reviews also state that technical solutions are not tested in practice or real-life tests [3], [41], [42].This can be an outdoor test and validation of medicine delivery or a multi-fire extinguishing task.The potentials of UAVs are not benchmarked against the default use, such as blood delivery by road and maternal healthcare provision by people [8].Additionally, studies often underreport aspects of their research such as flight information, environmental factors [25], [29], [30] and which software was used [35].Finally, a cost-benefit analysis or economic perspective to the study is often not executed [29], [42] even though the primary reasons to use a UAV are their accessibility and low cost [40].

A. UAV RESEARCH QUALITY
In the past decade, diverse studies identified, summarized and assessed the role that UAVs have played, and are playing in their research field.Despite the success of the technology itself, the quality of SLRs assessing the technology is lower than in other research fields.
The tertiary study on Machine Learning for Software Engineering [12], shows an average quality score around 3, rated on the same criteria as this TLR.Whilst the other UAV-related TLR scores a 0.3 [13], although missing one of the four DARE criteria in their assessment.Even if this fourth criterion would be a perfect score of 1, it would make it at best a 1.3 out of 4. This TLR presents an average quality score of 2.13, which is slightly higher than the TLR in [13].Similar to findings in other TLRs [9], [13]: we found that SLRs are not performing a quality assessment on the primary research (see figure 6).Due to this missing information, little can be concluded about the overall quality of primary research in UAV studies.
However, table 5 indicates that SLRs are not satisfied with the primary research.Methodological errors were identified, validation steps were missing, no comparison versus other methods was performed, relevant variables were not reported, and cost-benefit analyses were missing.This is a grim reflection on the quality of UAV research.However, the trend of quality in reviews is slightly upward over time (6), and much improvement is to be gained if SLRs are going to perform quality assessment [9], [12].
In the supplement of this TLR, the QA questions and rating schemes from [23], [24], and [43] are provided (See supplement III.quality assessment: UAVQualityAssessmentExamples.xlsx).Additionally, the strict criterion that [31] presented in the study on deep learning can also be found in this supplement.These examples could be directly used or adapted to suit the topic of the SLR.The reader is encouraged to take a look at this supplement for identifying important features in a UAV study, even if they are not planning an SLR.

B. REPORTING UAV RESEARCH
Underreporting of variables has been identified as a problem in primary research dissemination by SLRs [25], [29], [30].The underreporting of variables is also identified in this TLR, only 21 out of 73 studies performed a systematic data extraction.Perhaps the underreporting in primary research is a cause for secondary studies not to perform a data extraction: there is no data to extract.As elaborated in section IV-D (UAV Software), this problem of underreporting software packages and software processing approaches is even more profound.As indicated in this section, covering a UAV study systematically is a difficult problem: there are many variables that influence the final result of the study.To reckon with this problem, UAV primary research needs to report more details, and secondary research needs to align these details.This could finally open up the comparative analysis between UAV studies that has been missing since the dawn of this research field [15].
Inspired by the questions provided by [25], and recurring reported values in the secondary studies, detailed tables are provided (See supplement IV. primary research: primary.xlsx.An overview of this supplement is given in table 6.Additionally, reviews identify the future direction toward Deep Learning methods.An additional table provides Deep Learning-related aspects, which must be taken into account for UAV studies, inspired by the critical assessment from [31].The variety of UAV studies should be embraced, these tables provide a basis and should be adapted to fit the needs of a UAV application and research field, such as adapting to UAV swarm studies, the combination of various sensing payloads, or autonomous parcel delivery experiments. For future SLRs, empty data-extraction tables are provided (See supplement V. secondary: secondary.xlsx).An overview of the contents of these tables is given in table 11.A review should at least cover metadata on the journal, year, title and perform a quality assessment.In addition, information on the UAV experiment, analysis method, UAV platform, and payload also ought to be provided.

C. CONSOLIDATING UAV TECHNOLOGIES
The rise of UAV technology has come from various different technological developments in the past decades, such as the continuous shrinking of electronics, higher capacity batteries and improvements in compute per unit power, algorithms to balance unstable aircraft, faster networking, and much more.
Simply combining these technologies does not form a UAV, it is the seamless integration of these technologies that enables flexibility and low cost to such a wide audience.Additionally, this seamless integration also includes regulatory and social aspects, such as reliability of flight (it does not stop or crash, goes to the home position with low battery) and conforms to regulations (weight limits, does not take-off in restricted areas unless a request is made to the authorities), whilst newer propellers are being developed to reduce the noise [45].There is a complex system visible in a UAV platform: a system where state-ofthe-art hardware is combined to satisfy social norms through software development.This results in a platform that is difficult to get right, only a handful of manufacturers are able to do so and these are by a large margin the most popular manufacturers of UAV platforms included in this study (see figure 8).While others have gone under or have consolidated into larger companies.Switching manufacturers is not a simple decision.Vendor lock-in increases [46], [47]: proprietary communication protocols are already built-in to sensors and accessories such as radio controllers and payloads; costs of retraining for switching platforms; and the state-of-the-art increasing their lead on open source alternatives.
The UAV platforms have enabled various payloads to take the skies.From the total of 188 payloads that have been reported, there are 46 unique ones, although imaging sensors are the most frequent.This creativity in UAV applications will no doubt continue as access to flying platforms increases.Additionally, improvements in sensor technology such as higher resolutions, faster measurement intervals, and increased measurement sensitivity will have a direct impact on the capabilities of UAVs: more and more is possible.
Finally, the SLRs indicate a promising future using Deep Learning for UAV research [31], [48], [49], [50], [51].These complex models offer various improvements in accuracy over previous approaches and are therefore seen as the future of the field.Realtime use-cases for UAV flight such as obstacle detection, accurate localization and mapping, and semantic task understanding are significantly improved by DL [4], [49].Additionally, it is also the post-processing of the acquired data for disease mapping [25], [31], road segmentation [30], or crop yield prediction [49] that gets an accuracy boost from DL.However, these methods also present new problems, such as model biases, lack of labeled datasets , ethical risks, and environmental and financial costs [52].
There is an ongoing improvement in UAV technology: flight duration increases, sensors improve and analysis methods evolve [53].Relevant technology may now become outdated the following year.However, mutual gain is easily achieved as long as a wide audience remains to have access to the state-of-the-art.This is done through open research: best practices are shared, hardware standards are developed, and software approaches are easily reproducible.
Luckily, the open-source community has already developed many UAV software packages for wider adoption in various applications.Providing two-way interaction between academia and industry for improving the platform [54].

VI. THREATS TO VALIDITY
The limitations of this TLR are identified using the Validity of Classifying Threats scheme from [19].These threats are

A. STUDY SELECTION VALIDITY
The strategies used to search and select studies in this research are linked to the possibility of excluding relevant studies.The chosen year range for the search (2012-2022), preferred digital libraries, search terms used, and the selection criteria may have caused some studies to be missed.Expected is that broadening the query from UAV to also include variations, such as RPAS, drone and UAS would have increased the total search results.Additionally, the selection criteria for 'based on a defined search process' excluded many UAV review articles.However, this was done purposefully to only include SLRs, instead of any review.Finally, forward and backward snowballing could have been used to increase the number of SLR sources, however, the included 73 articles already covered a large variety of research fields, and there was a lot of overlap between the database results.

B. DATA VALIDITY
Acquiring different hardware, software, and themes from UAV studies requires many relevant variables to be checked and included.These variables are taken from the SLRs which performed a data extraction.This was the basis for a data-extraction schema.The used data-extraction schemas have been checked by the authors before starting the data-extraction process.However, many included articles did not provide a clearly structured table of their primary sources and the extracted variables, which are therefore not represented in the data extraction.For those that did present clearly structured tables, variation exists between study how details were presented.The schema was either adapted or the variable was adjusted: renaming variables to be more specific than originally presented: DJI P4 to DJI Phantom 4 Pro; Multirotor to Quadcopter (if the model was known).Finally, UAV technology moves rapidly, new hardware designs, features, and software updates make many older designs irrelevant and outdated.Therefore, using the information presented in the data-extraction tables to find the best hardware and/or software is discouraged.

C. RESEARCH VALIDITY
The research validity of this TLR is related to the generalizability of the results from our tertiary review to the population under study.Therefore, the representation of the selected articles should reflect the subject population.We tried to minimize this threat by searching different digital libraries and selecting all the results from the query, with a selected starting date of 2012, which is a meaningful starting date for the growth of UAV research, the oldest included article was published in 2016.The search, data extraction, and quality assessment have been performed by a single person which possibly introduced bias in these tasks, the resulting data extraction has been checked by all authors.

VII. CONCLUSION
This article presented a tertiary study on UAV applications.Based on 73 systematic literature reviews published between 2012 and 2022.This TLR aimed to assess the quality of SLRs, aggregate data across research fields,and provide broad guidelines to be applicable across the widespread UAV research community.The research questions supporting this aim are answered below.
RQ1.What is the quality of systematic literature reviews conducted in the UAV domain?
Quality assessment was performed on the included articles, based on criteria presented in [11].This assessment revealed a mix of higher and lower-quality research.The average score of the SLRs was 2.13 out of 4. The lowest-scoring criteria was the criteria covering whether a review performed a quality control step in their SLR, many SLRs did not perform one.Therefore, four different example quality assessments from the SLRs are presented in supplement III, as a guideline or for direct adaptation in UAV studies.
RQ2.What are the popular UAV platforms, payloads, and software in the UAV domain?21 out of the 73 articles performed detailed data extraction, covering hardware and software in UAV research.This extraction is performed to varying degrees of detail, where a low level of detail occurs more often.There is high creativity in UAV primary research, as the flexibility of the UAV enables many applications to take the sky.The quadcopter is by far the most popular flying platform, followed by the other multi-rotors, fixed-wing aircraft are also experiencing stable popularity, for certain applications hybrid-wing aircraft could also see growth in the coming years.There is a dependence on only a few manufacturers for supplying the flying platforms that are used to perform research.The further implications of this dependence vary in scale but are deserving of further attention.Payload-wise, if it is light enough to be mounted on a UAV, research is finding a way to make it fly.The different imaging sensors are the most popular, with the standard digital camera being the most popular.In the case of software, an important distinction needs to be made between the software package and the data processing approach.A software package is a set of libraries and executables to run a specific task.Whereas a data processing approach is the theoretical set of formulas and calculations processed in the software package.Both of these software-related topics are underreported in the SLRs.Therefore, little can be concluded about the popularity of specific packages or data processing approaches.Broad categories of used software packages fall under flight control, flight planning, image editing, sensor pre-processing, photogrammetry, geoinformation, and programming languages for analysis.
RQ3. Which limitations for using UAVs are identified in the SLRs?
The limitations of UAVs are social, technical, regulatory, and research-based.Where social limitations can be the negative perception of UAVs and the noise a UAV makes.The regulatory limitations are the lack of regulations and  How can future UAV studies be conducted to improve the quality of optimizing information sharing for practitioners and researchers?
Before UAV research can reach maturity, it needs to recon with its pubescent years.The first years of explorations and boundless possibilities are coming to an end, and UAVs will be mainstream technologies in the not-too-distant future.However, it is the improvement of research quality, technology, clear regulations, and social acceptance that are the roadblocks in getting UAVs of age.Therefore, a conclusion to the main research question is divided into advice for researchers and practitioners: Recommendations for Researchers Secondary research could improve significantly by systematically assessing primary research.Quality of research is not particularly high, in both primary and secondary research.Additionally, systematic data extraction should increase in detail, these details are what could enable larger comparative analysis, which is currently missing in UAV research at large.Provided in the supplement of this review are quality control questions, and data extraction tables, which are adapted from the included reviews.These should serve as a guideline for future research.The reader is therefore encouraged to take a look at these tables, to understand what ought to be reported in primary and secondary UAV research.Additionally, research fields should continue to devise and improve classification schemes and preferred reporting items for UAV research, improving adoption and replicability, whilst the technology continues to permeate across academia.Finally, UAV technology keeps rapidly improving, open source hardware and software should continue to be developed in research, ensuring generalization, the adoption of research, and mutual benefits of the rapid pace, whilst maintaining some independence from monopolizing companies.

Recommendations for Practitioners
The UAV practitioner is a concept that will grow in the coming years, as the adoption of UAVs will spread from research to industry, and more and more applications will see the rise of the flying platform.There is a variety of existing, viable UAV applications that do not cross paths with academia, such as professional photography and filming, drone racing, and a real-time eye-in-the-sky for police forces.Increasing collaboration with academia could provide fruitful in addressing the social and regulatory limitations of UAVs, through advocacy of more lenient and clear regulations and showing societal relevance.Additionally, regulations can also cover manufacturers to develop safe systems, by taking advantage of the intelligence built into the platform.Examples can be required obstacle avoidance, battery life notifications, and built-in geofencing before it can be sold.There is diverse research available on various UAV practices and analysis methodologies, practitioners can adapt these to develop real-world use cases of UAVs and seek out market niches.Traditionally, UAV platform and payload manufacturers have been providing steady improvements on the flying platforms such as reducing costs and increasing the capabilities, these improvements have enabled wider audiences to reap the benefits of the platform.To increase adoption and reduce technical limitations, this trend should persist in the coming years.Ideally, manufacturers increase their commitments to open-source hardware and software and create business models to support this change.Collaborations between industry and academia are luckily already prominent in the field and should continue to do so in the form of research projects, workshops, internships, field trials, and involvement in education [12].

APPENDIX A SUPPLEMENTARY MATERIALS
The supplementary materials can be found at: https://doi.org/10.5281/zenodo.7915562

APPENDIX C OVERVIEW OF SECONDARY STUDIES
See Table 12.

FIGURE 1 .
FIGURE 1. PRISMA inclusion/exclusion criteria chart.Exclusion reasons are presented in columns on the right side: duplicate records, language, not available, and no clear search query in review.Resulting in the 73 included SLRs.

•
Title and source (Journal article, conference proceedings) • Journal title • Number of included primary articles • Review topic (Summary of topic based on the article title)

FIGURE 2 .
FIGURE 2. Data extraction flowchart, spreadsheets were used to track extracted variables from the SLRs: (metadata, QA scores, and aggregations: hardware (payload and platform), software, limitations).

FIGURE 3 .
FIGURE 3. Year of publishing for included articles.

FIGURE 4 .
FIGURE 4. Topics of included SLRs over time.For visual clarity, the figure only covers the topics with more than one included SLR on the topic.

FIGURE 5 .
FIGURE 5. Histogram on the overall quality of included SLRs.

FIGURE 6 .
FIGURE 6.Average QA score grouped by year and averaged over the score.Individual articles can have higher scores than shown here.

FIGURE 7 .
FIGURE 7. Platform occurrence by year and type.Data is based on the release year of the primary source and the UAV type.SLRs not indicating details on UAV model or type have been aggregated to multi-rotor.

FIGURE 8 .
FIGURE 8. Popularity of UAV platform brands, based on the occurrence in primary articles, sorted by top 10 occurring brands.'Others' is the combination of all other brands not included in this top 10, consisting of 118 different UAV Brands.

FIGURE 9 .
FIGURE 9. Payload occurrence by year and type.

FIGURE 10 .
FIGURE 10.Popularity of payload brands, based on the occurrence in primary articles, sorted by top 10 occurring brands.'Others' is the combination of all other brands not included in this top 10, consisting of 27 different UAV Brands.
subdivided into Study selection, Data validity, and Research validity.The threats are covered under their respective section, with the details and mitigations as tables.

FIGURE 11 .
FIGURE 11.Visual overview of included SLR articles.Different colours correspond to different topics in UAV-related research.The italic article shorthands are the reviews which provide detailed information on hardware or software used in the primary studies (n=20).The bold text are rated a quality score of 3 or higher (n=16).The left figure shows the start and end date of the search query of the SLR.And the right figure are the number of included articles that the SLR identified.

TABLE 2 .
Reviews with full quality assessment.

TABLE 3 .
Reported hardware items with varying levels of detail.

TABLE 4 .
Software packages reported in the included SLRs.

TABLE 5 .
Recurring limitations indicated in UAV SLRs.

TABLE 6 .
Overview of parameters to report on in a UAV primary study.

TABLE 7 .
Overview of data extraction parameters to report on in a UAV secondary study.

TABLE 8 .
Study selection threats and mitigations this study.

TABLE 9 .
Data validity threats and mitigations in this study.

TABLE 10 .
Research Validity threats and mitigations in this study.

TABLE 12 .
Included studies, ordered from high to low based on quality assessment score.

TABLE 12 .
(Continued.) Included studies, ordered from high to low based on quality assessment score.