The APISSER Methodology for Systematic Literature Reviews in Engineering

[Background] Every research topic is first to be addressed by understanding its current State of the Art. A Systematic Literature Review is a trustworthy method for establishing the published State of the Art of any given topic. In engineering sciences, we have failed to consistently, methodologically, and thoroughly execute Systematic Literature Reviews at the beginning of every research path, and to standardize the method to do so. Currently available methodologies fail to link a method to a customized and much-needed tool support. If the high-effort demanding task of executing a Systematic Literature Review is not well tool-supported, it will soon become manually unmanageable to handle the large amount of data involved. [Objective] Therefore, we want to take a step forward towards standardizing the methodology for executing Systematic Literature Reviews in engineering by proposing a tool-supported task-oriented engineering flow method to execute Systematic Literature Reviews in engineering. [Method] Based on the well-known and proven Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology from medical sciences, we adapted and enhanced the method to follow a task-oriented engineering flow and to be supported by customized tools. [Results] In this paper, we first present the APISSER methodology for Systematic Literature Reviews in engineering, and then show its practical tool-supported application in a case study. [Conclusion] We have shown how the method successfully results in the collection of valid literature to report a trustworthy published State of the Art in engineering sciences.

T HIS paper presents the APISSER methodology to conduct a Systematic Literature Review (SLR) in engineering, and it is organized as follows: Section I will provide the context and motivation (rationale) that drove us to propose the methodology. Section II presents a brief background of the existing methodologies for literature reviews and how this work builds upon them. Section III presents the APISSER methodology. Section IV shows a case study where we successfully applied the method. Finally, section V concludes the paper and provides an outlook.

I. INTRODUCTION
Engineering research follows the scientific method. This method involves an initial observation of a knowledge gap or problem, a clear motivation to address it, and the statement of Research Questions (RQs). After this initial stage, we first need to establish what has been done so far in the field of interest, that is, to establish its State of the Art (SoA). There-fore, unless we are leader authors in the field of interest, it is a requisite to conduct a literature review. The latter implies going through existing relevant literature of a reasonable time frame in the past on a specific topic. The number of publications to review will depend on the innovative character of the topic we are dealing with and the timeframe of interest, but most likely the number of publications will rapidly explode. If we do not methodologically address this process, it may result in the reviewers 'drowning in a sea of literature' or not covering it thoroughly. Once a literature review has a defined, auditable and reproducible methodology, it becomes a SLR.
The 'diving into a sea of literature' to establish the SoA of a given topic is a real challenge that every scientist faces at the beginning of a new research topic. That is where SLRs as a literature genre [1] represent a check-point in time of human knowledge for a specific topic and RQs. In the best case, a recent literature review already exists or it needs to be updated [2]. But if it does not yet exist, it must be conducted by the research team at the beginning of the research cycle. Unfortunately, it is often skipped or executed in a superficial, incomplete, and nonsystematic manner. In the latter case, the scientific method is incomplete, as the SoA has not been properly established. Therefore whatever research is carried out, does not necessarily extend previous work and knowledge. As scientists, we need to approach every step of our research in a methodological, reproducible, and efficient way.
The query of the term: 'Systematic Literature Review' in the Web of Science (WOS) Data Base (DB) (Figure 1) shows engineering sciences ranking 13th in research categories where SLR are found, and a clear dominance is shown by medical sciences. In medical sciences, the execution of SLRs is very common, as it is a trustworthy method to make evidence-based decisions that directly affect the life of patients. Therefore, they have developed a clear methodology to guide its execution: the PRISMA methodology [3]. Since its inception in 1999 [4] and further development, the methodology has proven to be trustworthy and efficient.
Technological advancements have also changed the way publication data is stored. Nowadays all scientific DBs have a web-based interface, and using a search query is the most common method for finding publications in a relevant field. When executing a literature review using a web-based DB engine, the results of a single query rapidly explode. The number of publications rapidly increases and soon enough it becomes humanly impossible to manually manage the amount of information that is obtained. This clearly points to the need of Information Technology (IT) tools to support the execution of a SLR.
In engineering, no standardized methodology exists for the execution of SLRs. The process to execute a SLR is well stablished, but we have found a loop hole in its practical execution linked to tool support. For that purpose, we propose to take the advantage of the widely proven PRISMA methodology and adapt it to a tool-supported, task-oriented engineering flow. Therefore, in this paper, we present the APISSER methodology, a spin-off of the PRISMA methodology, which proposes a tool-supported task-oriented engineering flow method to execute SLRs in engineering.

II. BACKGROUND
Performing a literature review is the action of searching through a trustworthy literature source for relevant publications that represent the published SoA of a specific topic. A SLR has the add-on of having a defined, auditable, and reproducible method for doing so. The results of a SLR will lead to Proof-of-Evidence (PoE) based decision-making, yet the review is only as good as the method behind it. The conceptual method behind executing such a review is well defined in literature, e.g. [6], [7], yet its application in the field of engineering and the tools to support it, are not yet clearly defined.
Medicine is the leading field in published SLR studies, because it is necessary to enable Evidence-Based Medicine  (EBM) [8]. This led to the standardization of the method in the PRISMA methodology [3]. In engineering, we have failed to adopt such a methodological approach and consistency in the execution of SLRs. Nevertheless, this type of publication has been increasing in the field of engineering over the past few years (Figure 2), which indicates the need for the standardization of the method. The first steps in defining a standard methodology were taken by computer engineering scientists when Kitchenham in 2004 proposed the Evidence-Based Software Engineering (EBSE) method [9]. In 2018, Torres [10] adapted the method to engineering and education. A recently released (2020) work-in-progress study [11] intends to identify whether SLRs in engineering follow a specific method, but we await further results. All these methodologies have advantages and disadvantages, but none of them are truly linked to a much-needed customized tool support. The need for automation (tool support) in SLRs is clear [12]. There are many commercial tools available in the market for a paying fee [13]- [17], as well as open-source web-based solutions [18]. However, most teams use these tools to support only certain parts of the execution of the SLR, but end up diverging at some point due to the tool not supporting exactly the method they intend to apply. We see the need for a customized open-source tool approach to fully meet the requirements of the research team and topic of the SLR at issue. All the previously mentioned tools have advantages and disadvantages, but none of them is truly linked to a methodology.
In the present work, we aim to link the methodology for executing SLRs to a customized tool support and to make this a task-oriented process, following an engineering flow. Therefore, we present the APISSER methodology: a spin-off of the PRISMA methodology applied to engineering sciences with a tool-supported task-oriented engineering-flow. This methodology includes a sub-set of the full PRISMA methodology to simplify the process and show that a basic SLR can be realized with a reduced number of steps, therefore encouraging the community to execute these more often at the initial phase of every research.

III. THE APISSER METHODOLOGY
The APISSER methodology is based on the PRISMA methodology. However, it is reduced and adapted to a taskoriented engineering flow and enhanced with tool support to efficiently manage the vast amounts of information. The APISSER method comprises six stages: A priori, Plan, Identify, Screen, Select, Extract and Report. The checklist for method is presented in Table 1.

A. A PRIORI
A priori, we must have identified a knowledge gap/problem to have a A1 (A1) topic, from which we want to establish its SoA. We must then justify the significance of addressing the topic/problem, which states a A2 (A2) rationale. Finally we need to clearly list the A3 (A3) RQs we will answer with our SLR.

B. PLAN
First, we define five criteria that will P1 (P1) characterize our study: the reach of the study (boundaries), keywords, type of publications, databases, and target journal. The reach of the study (boundaries) clearly shows the assumptions and limitations of our study. The keywords is a list of terms that completely cover the topic and RQs. These keywords will then be used as terms in the search queries that we will execute in the DBs to identify relevant publications. The type of publications is to be a selection of the following list: journals, conferences, patents, books, dissertations (bachelor , master , doctoral). The electronic DBs to be used are defined and its search syntax should be reviewed. For topics in engineering we suggest the use of the Web of Science [5] and Scopus [19] DBs. If patents are part of the research scope, we suggest using the European Patent Office [20]. We find an appropriate journal for later publication of our findings. This previous step is important because our publication should fit the intended journal; the latter may help to better formulate RQs and adjust the characterization of the study. We recommend Elsevier [21], Wiley [22] or Institute of Electrical and Electronics Engineers (IEEE) [23] online publication finders.
From the characterization of the study, we can derive the P2 (P2) Eligibility Criteria (EC) that must be fulfilled by the publications included in our study. These are of two categories: inclusion and exclusion. The Inclusion Criteria (INC) is a set of numbered qualifications that the publications included in our study must fulfil. The Exclusion Criteria (EXC) is the set of characteristics that we want to avoid in the publications selected for our study, which will ease the process of rejecting publications. We recommend the definition of tags, which will help the reviewer determine the inclusion or exclusion of a publication. The tags will, later on, be color-highlighted on specific information fields of the publication in the tools used by the reviewer to execute the screen & select and extract phases. We now define and number the P3 (P3) data items [Dxx] to be extracted from the selected publications. Each data item to extract is to be linked to a specific RQ. The possible values and data types for each item are also be defined.
We define the P4 (P4) DB search strategy, that is: to define the method by which we execute the search queries in the DBs (manually vs. automized) and define the stop criteria that identifies the stage when sufficient search queries have been executed. We then define the P5 (P5) selection process, through which we methodologically prove the EC to identify the selected publications. This process describes the actions to take in the identify, screen & select, and extract phases. We also define P6 (P6) tools and data management, which supports the execution of the selection process as it will become manually unmanageable. We recommend the development of two tools, one for the screen & select phase and the other for the extract phase, and these to be built using Python [24]. We need to define the DB structure (tables-fields), and the required software: the number of tools, their Graphical User Interface (GUI) and the method by which these tools will access the main DB to display publication data, and fill the INC and data items for each publication.
All the information, data, and plan generated until now must be collected in a document called the P7 (P7) SLR protocol. The use of a project management workflow is recommended to assign tasks and schedules to the team. This document should be submitted for internal review and agreed upon by all parties involved. VOLUME x, 2021

C. IDENTIFY
From this point onwards, we will start to manage a large amount of information and there will be much human effort needed, as it is time to 'dive into the sea of publications' to extract the ones that are relevant for our study. Reviewers will be needed, but they will be supported by automated tools that will ease the organization, storage, and processing of information, thereby accelerating the entire process. We encourage the management of all data processing with a highlevel general-purpose script-based programming language, we highly recommend Python [24].
Using the previously defined keywords, we conduct I1 (I1) DB search queries. The search strategy that we took is the manual 1 execution of search queries in the previously defined DB until 2 a reasonable logical combination of ALL keywords with some selected INC and EXC terms was achieved.
From each search query, we proceed to I2 (I2) export publication data in a machine-readable format for further processing; we recommend the Comma-Separated Value (CSV) format. We remind the reader that the fields exported from each publication are DB dependant. The exported data of the publications is to be I3 (I3) built into a Local Data Base (L-DB), for example the SQLite DB [25] format. We recommend importing each search query into a table in the L-DB to maintain order. A machine-readable L-DB file is to be created and a version control system used to back it up and track changes; we recommend the Global Information Tracker (GIT) open-source version-control system [26]. After loading the search query results in the DB, each item (publication) is to be extended with the columns shown in Table 2. The field REP should be marked in all publications at this stage, based on wether it is repeated in-between search queries of our study, the latter can be done script-based. Repeated publications are to be reviewed only once. The L-DB can be visualized with any DB browser, for example, the DB Browser for SQLite [27].

D. SCREEN & SELECT
In this phase, we aim to determine and fill the IN x in the L-DB for each publication, in order to determine whether it will be included or excluded from our study. This phase of the method is better supported by a GUI. In this GUI the publication data is to be displayed, and the previously defined tags in the selected fields are to be color-code highlighted. The GUI helps the user to better visualize the information of 1 We hypothesize that the search process in the DBs can also be managed using a script that opens a connection to the DB server. As an example, we found libraries that can access the WOS DB using a Python script, but this process is yet to be evaluated. We highly encourage this because it automates the search process. 2 The algorithm used to determine when enough search queries have been executed is still an open topic in this methodology; that is, the algorithmicbased search strategy is yet to be defined, opposed to the manual combinatorial approach adopted here. The algorithmic approach might be based on the criteria of achieving all reasonable logical combinations of the keywords, in combination with the information of the number of repeated publications found in each successive query. The screen phase has two stages, a screen-basic and a screen-overview stage. The INC are to be split into two subsets: one to be checked during the screen basics stage and the other to be checked during the screen overview stage. In the S1 (S1) screen basics process, the reviewer's stage is to check the first set of INC by reading only three items from each publication: title, author, and abstract. The publications that comply to the first set of INC, will move forward to the We have now selected the publications that will be part of our SLR study.

E. EXTRACT
Finally, the selected publications move forward to the E1 (E1) data extract phase, in which the reviewer identifies and extracts all previously defined data items [Dxx] for each publication. It is also recommended that a categorization of the publications be made at this point, for better sorting of the information. A limited number of categories are to be defined, and the reviewer should assign for each publication one of these categories. This category is to be recorded for each publication in the SLC field of the L-DB. This is manual work in which a reviewer is required to fully read through each publication to determine the value of each data item and fill it in the L-DB. The knowledge gained by the reviewer will then mold the analysis of the extracted data and discussion in the report publication. The reviewers should be the authors of the report publication. This phase of the method is better supported by another GUI to display basic data of the publication and to write the data items to the L-DB.

F. REPORT
We then R1 (R1) write the publication to report our findings. This should be based on the requirements, format, and structure imposed by the selected journal or conference, and using academic writing practices and guidelines 3 . The focus of the publication should be on answering the initial RQs in combination with an analysis of the collected data items; however, any work can be extended after that initial aim. Inspired by the recommended report items of the PRISMA methodology [35], [36], we recommend that the SLR report has the following structure 4 : Title, Abstract, Keywords, Introduction, Method, Background, Results, Conclusion, and References.
The publication draft should undergo an internal R2 (R2) review process. We recommend a technical and an Englishproofread review. There are online tools for English language polishing [37], [38]. Finally, we R3 (R3) summit the publication to the journal for publication.

IV. PROOF OF CONCEPT: A CASE STUDY
We applied this methodology in [39]. In the following sections, we summarise the elements of the methodology for that publication.

A1 (A1) The topic of our SLR is the review of Electronic Control
Systems (ECSs) for Ion-Trap Quantum Computers (ITQCs). A2 (A2) As we upscale ITQCs, the ECS must efficiently scale and adapt. With this SLR we aim to have a clear understanding of how these ECSs have been designed so far, so that in future research we may propose an efficient system-level architecture for the ECS of an ITQC. A3 (A3) We will address the following RQs: RQ0. What is the use-case for the ITQC? RQ1. What are the physical requirements of the ECS? RQ2. What are the electrical requirements of the ECS? RQ3. What 3 We highly recommend using L A T E X [28], [29] as a software system for document preparation to report every publication in research. A template is typically available at the journal/conference or can be generated [30]. In [31] some recommendations on how to organize a research publication can be found. The language should be academic English; some valuable recommendations and guidelines can be found in [32]- [34]. 4 The Title should have either one of the following structures: (1) A Systematic Literature Review on topic or (2) Topic: a Systematic Literature Review. The Abstract should include the following sections in brackets: Background, Objective, Method, Results, Conclusion. The latter is to ease readibility, which is a common practice in medical sciences. The Keywords is the list of previously defined index items. The Introduction section should include the 'A Priori' items (A1-A3). The Method section should include the 'Plan' items (P1-P4). The Background section provides a basic overview of the field, and/or points to a previous SLR. The Results section classifies and displays SLR's findings. The Conclusions section provides an outlook of the work. The Reference section lists all selected publications, and other relevant literature used. system-level architecture and technologies are used in the ECS? 5

P1
(P1) Characterisation of study: The reach of the study of the SLR is to find publications that describe an ITQC with (at least) minimal included information about its ECS design, implementation, and/or verification. We focus on Paul trap-based ITQCs with a Quantum Charge-Coupled Device (QCCD) architecture and the required shuttling of ions. The keywords that will drive our study are quantum, computer, control, electronics, ion trap, shuttling, cryogenic, and vacuum. The types of publications we will evaluate are Journal Articles and Conference Proceedings Articles. The online DB we will use to execute our search queries is the WOS [5]. The target Journal for the publication of this SLR is: IEEE Systems Journal [40]. The P2 (P2) EC is composed of the INC shown in Table 3, and the EXC shown in Table 4. The tags, their category and colour-code are also shown in Table 4. The P3 (P3) Data items to extract are listed in Table 5. The P4 (P4) DB search strategy we took was to manually execute search queries in the previously defined DB until a reasonable logical combination of ALL keywords with some selected INC and EXC terms was achieved. The P5 (P5) Selection process is shown in Table 6. As for the P6 (P6) Tools needed, we will develop two GUIs in Python [24]. One to support the screen & select phase (ss_gui.py), and the other to support the data extract phase (ex_gui.py). According to the steps in the selection process, the data management proceeds as follows. From each search query, a full record of all identified publications is exported from the WOS DB in CSV format (wos*_raw.csv). Using Python scripts, the data are parsed, imported, and organised in tables of a SQLite DB [25] (slr.db). We extend each item (publication record) in the database to add the extended fields listed in Table 2. This extended DB is our starting data pool and is referred to as L-DB, from which the GUIs will read and write data. Through scripts, we recognize repeated publications and mark accordingly the field REP in the L-DB. We use the ss_gui.py GUI to fill the IN x, SBN, SON fields of the L-DB. Publications that comply with all INC (i.e. IN x == true) are denoted as SEL = true. The ex_gui.py GUI will then be used to fill the Dxx and SLC fields. Using Python scripts, we will access the filled DB and generate a report of the selected publications; all the data collected will then be available for the report. P7 (P7) All previous information was collected in a SLR protocol and was sent to the parties involved for review. 6

C. IDENTIFY
We start now the execution of the the selection process, described in Table 6. In this phase we perform I1 (I1) DB search queries. The search queries performed in the WOS    lication data by using the export function of the WOS DB. We generated and downloaded a CSV file from each query, which includes the publication data of the results. We I3 (I3) build the L-DB by loading all CSV files into it, each search query in a table, and extending the fields according to Table 2. The generated DB slr.db is illustrated in Fig. 3.

D. SCREEN & SELECT
As planned, we developed the ss_gui.py GUI 8 , as shown in Figure 4. It shows all relevant fields of each publication so that the reviewer can easily determine and mark the inclusion criteria. The tags depicted in Table 4 are highlighted in the selected fields.
In the S1 (S1) Screen basics phase, solely by looking at the title, abstract, and keywords of each publication, the reviewer determines IN 0 − 5 9 for that publication. The S2 (S2) Screen overview phase allows the reviewer to determine IN 6 − 9 by performing a quick overview of the paper. Finally, we can script-check the L-DB to look for the publications in which IN 0 − 9 are all true, namely, these are the S3 (S3) selected publications for our study and their SEL field is to be marked as true.

E. EXTRACT
In this phase, we proceed to extract the E1 (E1) Data items [Dxx] only from the selected publications. The data items are filled by the reviewer as he/she reads through the full publication. As planned, we developed the ex_gui.py GUI 8   as shown in Figure 5. The tags listed in Table 4 are again color-code highlighted in the selected fields.

V. CONCLUSION
We have shown the need to standardize the methodology to execute SLRs in engineering. Following the steps of our medical science colleagues, we have taken the effective-proven PRISMA methodology and developed a spin-off version: the APISSER methodology, which is enhanced by a taskoriented flow and tool-supported approach. We have shown how this method can be applied to the successful conception, development, and publication of a SLR in engineering.
We encourage the use of this method as it best fits the research team. Furthermore, we encourage future work, extending this initial work, towards the standardization of the method to execute SLRs in engineering. We trust that the present work will encourage engineers to execute a SLR at the beginning of every research process, given that the failure to do so can turn out into not building on each other's work and knowledge.