A Systematic State-of-the-Art Analysis of Multi-Agent Intrusion Detection

Multi-agent architectures have been successful in attaining considerable attention among computer security researchers. This is so, because of their demonstrated capabilities such as autonomy, embedded intelligence, learning and self-growing knowledge-base, high scalability, fault tolerance, and automatic parallelism. These characteristics have made this technology a de facto standard for developing ambient security systems to meet the open and dynamic nature of today’s online communities. Although multi-agent architectures are increasingly studied in the area of computer security, there is still not enough empirical evidence on their performance in intrusions and attacks detection. The aim of this paper is to report the systematic literature review conducted in the context of specific research questions, to investigate multi-agent IDS architectures to highlight the issues that affect their performance in terms of detection accuracy and response time. We used pertinent keywords and terms to search and retrieve the most recent research studies, on multi-agent IDS architectures, from the major research databases and digital libraries such as SCOPUS, Springer, and IEEE Explore. The search processes resulted in a number of studies; among them, there were journal articles, book chapters, conference papers, dissertations, and theses. The obtained studies were assessed and filtered out, and finally, there were over 71 studies chosen to answer the research questions. The results of this study have shown that multi-agent architectures include several advantages that can help in the development of ambient IDS. However, it has been found that there are several issues in the current multi-agent IDS architectures that may degrade the accuracy and response time of intrusions and attacks detection. Based on our findings, the issues of multi-agent IDS architectures include limitations in the techniques, mechanisms, and schemes used for multi-agent IDS adaptation and learning, load balancing, scalability, fault-tolerance, and high communication overhead. It has also been found that new measurement metrics are required for evaluating multi-agent IDS architectures.


I. INTRODUCTION
The openness and dynamic nature of today's online communities have raised a lot of security concerns on our digital valuables. Intrusions and attacks are intentionally The associate editor coordinating the review of this manuscript and approving it for publication was Muhammad Khandaker . designed to corrupt information and network communications. Recently, they have been rapidly increasing due to the extensive use of computer networks. A new report released by Semantic says that they record events from 123 million attack sensors worldwide, block 142 million threats daily, and monitor threat activities in more than 157 countries [1].
For protecting computer networks, Intrusion Detection System (IDS) is used to detect intrusions and attacks in realtime by analyzing network activities using statistics [2]- [4], rules [5]- [8] or machine learning [9]- [11]. IDS can be hostbased, network-based, or hybrid. Host-based IDS is used for monitoring and analyzing the internal computer system state [12], while network-based IDS is to monitor and analyze the external computer system state(network traffic) [13]. The hybrid IDS combines both host-based and network-based. After the expansion in computer networks usage, the traditional host-based and network-based IDSs that use statistical, rule-based, or conventional machine learning methods had been ineffective in detecting the massive, sophisticated attacks that invade computer networks in extremely high speeds. As a result of this, significant progress has been made in IDS research by moving from the traditional methods towards more intelligent techniques.
Several multi-agent IDS architectures have been proposed using machine learning and other advanced computational intelligence methods [14]- [20], [28]. The main difference between the proposed multi-agent IDSs and the traditional distributed IDSs is that multi-agent IDS deals with distributed problem solving and how agents interacts to detect attack incidents in computer networks, while the traditional IDS is about the distribution of the IDS architecture itself [21]. Besides that, multi-agent systems own unique capacities such as autonomy, portability, mobility, and social capabilities [22]. All these capacities motivated researchers to use this technology as it is suitable for solving the complex tasks of intrusions and attacks detection especially in the open and dynamic online environments [33].
Although the multi-agent architectures are increasingly studied in the area of computer security, but still there is no enough empirical evidence on their performance in intrusions and attacks detection. This lack of evidence limits the utilization of multi-agent technology in IDS research. By conducting this SLR, we will provide an up-to-date comprehensive reference for IDS researchers and developers to start new research and utilize the best techniques in the literature.
The aim of this systematic literature review (SLR) is to investigate the most recent multi-agent IDS architectures to highlight the issues that may affect the performance of intrusion detection in terms of accuracy and response time. To achieve this aim, we focused on four aspects of multi-agent IDS architectures: (1) The classification of multi-agent IDS architectures, (2) The aspects that influence intrusion detection performance in terms of accuracy and response time, (3) The limitations of these aspects (4) The measurement metrics that are used to evaluate multi-agent IDS architectures.
This SLR was carried out in four phases, firstly: a set of research questions were formulated based on the study aim. Secondly, search processes were launched, and materials were collected. Third, the collected materials were assessed and filtered out to choose the most relevant studies for providing answers to the research questions. Finally, data from the selected studies were synthesized and compared according to the formulated research questions. This SLR is limited to answer the formulated research questions. The data collected is related to the selected studies that were chosen using the selection criteria stated in subsection III-A3. This SLR is not concerned with the traditional host based and distributed network based IDS.
The remaining parts of this paper are organized as follows: Section II reports related works on multi-agent IDS literature review. Section III describes the methodology followed in carrying out this research. Section IV presents the discussion of the results based on the formulated research questions. Section V summarizes the publication limitation. Section VI is the conclusion of this study.

II. RELATED WORK
According to our investigation, to date, there is no SLR conducted on multi-agent IDS architectures. However, a limited number of literature reviews have been reported in this domain. The report in [23] discussed and summarized the use of mobile agents in intrusions detection along with their advantages and disadvantages. The advantages include reducing network latency and traffic, asynchronous execution and autonomy, structure and composition, dynamic adaptability, dealing with heterogeneity, fault tolerance, and scalability. The report also raised issues on mobile agent security, performance, code size, lack of a priori knowledge, and coding and deployment difficulties. The study in [24] investigated the immunological essentials in designing a multi-agent IDS. The early five autonomous agent architectures for distributed intrusion detection were evaluated. The literature review in [25] investigated network and agent based IDSs. The first part of the review focused on the network based IDSs while the second part shed the light on IDS based on mobile agents.The review enumerated the architectural characteristics and advantages of using mobile agents, and discussed three of the agent based IDS architectures. The study in [26] discussed multi-agent IDS architectures based on immune system algorithms. The study focused on agents' roles, architectural characteristics, and the security mechanisms used for securing computer network. The survey study in [27] reviewed the existing trends in IDS. Beside highlighting the advantages and disadvantages of the data mining and soft computing techniques used in intrusions detection, the survey also discussed agent based IDS, honeypots and honeynets. The literature review in [28] investigated the multi-agent IDS from an architectural point of view. There were approximately 15 studies related to IDS based on stationary agents and 15 studies related to IDS based on mobile agents. The review focused greatly on the types and distribution of the agents used. Also, the review discussed how a multi-agent IDS is constructed and how data flows.
The review paper in [29] presented a classification for the typical IDS and then conducted a strategical review on the existing mobile agent-based IDSs focusing on their classification, architectures, mode of data collection, data analysis techniques, and their security.
In our previous paper [30], we studied the evolution of malware detection systems from an architectural perspective and detection techniques used. The study also highlighted the importance of agent-based architectures in the domain of IDS. The review paper in [31] presented a classification for agent types, and the advantages and disadvantages of using agents. The review summarized agents' advantages in: the asynchronous autonomous interactions, reduction of network load, dealing with heterogeneity and ease of configuration. On the other hand, this review summarized the disadvantages of using agents in security issues and absence of common language. The review paper in [32], focused on classifying the existing wireless IDS techniques based on target wireless network, detection techniques, data collection process, trust model and analysis techniques. The pros and cons related to four of the proposed architectures were highlighted. These pros and cons that concern agents' interaction, coordination, management and data analysis.
The literature reviews reported above do not provide comprehensive details on the characteristics of multi-agent architectures that improve the performance of intrusions detection. There are essential characteristics that need to be embedded in multi-agent IDS architectures to improve the performance of attacks detection such as self-learning, adaptation, scalability, load balance, fault tolerance, self-management, self-configuration, and robustness.
This review provides new in-depth analysis for the major properties and characteristics that greatly impact intrusion detection performance. We highlighted the most important limitations of these properties such as multi-agent organizational structure, and computational components (data collection, data synchronization and data analysis). We also outlined the shortcomings of multi-agent IDS properties and characteristics. The finding of this review have not been addressed in any of the other reviews.

III. RESEARCH METHODOLOGY
A systematic literature review (SLR) is a specific research methodology used by researchers to gather and evaluate the evidences related to the topic under investigation. The protocol developed in [34], for conducting a systematic literature review in software engineering was utilized in this SLR to design the research plan described in the following subsections.

A. RESEARCH PLAN
The plan of this SLR consists of four phases: In phase 1, the research questions were formulated In consistency with the main goal of the study. In phase 2: search strategy was determined to specify search elements such as choosing search keywords, defining search strings, and determining electronic resources. This phase also included the tasks of choosing a reference management software tool, executing search processes, and collecting studies. In phase 3, we execute the selection policy that prepared for choosing the relevant studies, and quality assessment. Finally, in phase 4, we analyze data synthesis. These phases are summarized and illustrated in Figure 1. The subsequent sections discuss each phase in more details.

1) RESEARCH QUESTIONS
This systematic literature review aims to investigate the current multi-agent IDS architectures to identify the most challenging limitations that affect intrusions detection performance. Four research questions were formulated as follows: RQ 1: What classifications exist for multi-agent IDS architectures? RQ 2: What aspects of multi-agent architectures influence intrusion detection performance in term of speed and accuracy? RQ 3: How the characteristics of multi-agent IDS architectures impact the speed and accuracy of intrusion detection? RQ 4: To what extent the metrics used to measure and evaluate multi-agent IDS architectures are suitable and sufficient? To ensure robust and precise research questions, we carried out two step-verification: firstly, the research questions were formulated by referring to the recommendations stated in [36]. Secondly, the formulated research questions were validated and cross-checked by experts in the same field. In this phase, we chose the search keywords and strings, determined the electronic resources, selected a reference management tool, and defined the search execution process. In the following subsections, each process is explained in detail.

a: SEARCH KEYWORDS AND STRINGS
The search keywords and strings were derived from the research questions of this SLR. Synonyms and alternatives of the keywords and terms were also included in the search keywords. The synonyms, keywords, and terms were taken from the relevant research papers in the field of multi-agent IDS.

b: ELECTRONIC RESOURCES
For retrieving the relevant studies, search electronic resources were determined. We chose to retrieve studies from journals, digital archives, digital libraries, and online bibliographic databases. Examples of these resources include ACM Digital Library, Springer, ScienceDirect, IEEE Xplore, Google Scholar, and Google search engine.

c: REFERENCE MANAGEMENT
A large number of studies were retrieved from the online resources by using the search strings and keywords. The retrieved materials were collated and organized by a reference management software called EndNote. 1 This made it easy, adding and removing the studies whenever it is required.

d: SEARCH PROCESS
The search processes were launched on online electronic resources to retrieve journal articles, conference papers, book chapters, and theses. The references were recorded, and the full pdf files were downloaded and stored. These search operations resulted in more than 1000 studies. The EndNote was used to combine each reference with its' related pdf file to make it easy to read the papers. Afterward, the study selection process was applied to filter out the unrelated studies.

3) STUDY SELECTION
To choose the most relevant studies, two procedures were conducted. Firstly, the studies' titles and abstracts were manually checked. The studies that matched the aim of the research were selected, while the other studies that do not match the aim of this SLR were discarded. This process resulted in more than 220 relevant studies. The relevant studies were collated, and their bibliographic information was checked. Secondly, study selection was performed by applying the 1 Software program for reference management inclusion/exclusion criteria described in Table 1. This selection operation resulted in more than 70 relevant materials. The search process and study selection are illustrated in Appendix G.
In addition to the described method for including and excluding studies, manual checking was performed on the selected studies, and more relevant studies were added or removed to the group. Also, studies with unknown reference sources were excluded. This process finally resulted in 71 studies, including 37 journal articles, 21 conference papers, 7 book chapters, and 6 theses. This is illustrated in Appendix H.

4) QUALITY ASSESSMENT
Quality assessment is an important step to show the reliability level of the studies under investigation. The assessment process was conducted based on two elements: first, the studies were assessed based on their reference type and grouped by their year of publication. We chose only the most recent studies with popular and indexed reference sources. Then, we evaluated the contents of the selected studies. In the following subsections, the assessment of the selected studies is explained.

a: QUALITY ASSESSMENT BASED ON REFERENCE TYPES AND PUBLICATION YEAR
In this step, the studies were evaluated using two measurements: material type and year of publication. First, after the selected studies had been chosen, their bibliographic information was checked. A lot of materials, including journal papers, book chapters, and theses, were collected. The sources of the collected materials were identified. The majority of the materials were indexed under the major indexing databases such as SCOPUS, ISI, and IEEE. Figure 2 shows the percentages of the selected studies per their indexing types.
Second, for sound review, the selected studies should reflect the state-of-the-art of multi-agent IDS. All the selected studies were from 2010 to 2019. Figure 3 shows the distribution of the studies per year of publication.
More details on reference and sources of the selected studies are shown in Appendix D, and Appendix E.

b: QUALITY ASSESSMENT BASED ON THE STUDIES CONTENTS
To evaluate the contents of the selected studies, we developed quality assessment questions to be answered by either ''yes'', VOLUME 8, 2020   ''no'' or ''somehow''. These questions are shown in Table 2. Each question must be answered by an option that has a number associated with it: ''yes'' = 1, ''no'' = 0, and ''somehow'' = 0.5. The total score for each study was computed by averaging all the scores. To ensure that the selected studies are reliable, we only considered the studies with scores above 50% because those rated below 50% are either conceptual papers or include frameworks for other papers that already included in the selected studies.
Using this method for weighting the selected studies is very effective in giving insights about the reliability of the studies' contents. The quality scores for the selected studies are shown in Appendix F.

5) DATA SYNTHESIS
In this step, the scrutinized papers were carefully reviewed, compared, collated, and summarized according to the formulated research questions. The papers' summaries include qualitative data such as characteristics, properties, and performance metrics of multi-agent IDS architectures. We organized these summaries in table formats to help to analyze and interpret the results.
Data related to the research question RQ 1 was extracted and organized in a tabular format. Appendix A illustrates the architectural characteristics and properties of multi-agent IDS, and Appendix B illustrates agent types exhibited by the selected studies. To synthesize the data related to the research question RQ 2, the aspects of multi-agent IDS architectures that influence intrusion detection performance were identified and placed in tables for coherent analysis. This is shown in Appendices B and C. The limitations of multi-agent IDS architectures that related to research question RQ 3 were extracted and organized in textual formats. The metrics used for evaluating multi-agent IDS architectures, question RQ 4, were enumerated in textual forms. Synthesizing the results data into tables would help in making coherent analysis and interpretation for the research findings.

IV. RESULTS DISCUSSION
The aim of this SLR was to investigate the current multiagent IDS architectures to highlight the issues that affect intrusions and attack detection in terms of accuracy and response time. To achieve this goal, the discussion scope was centred around answering four research questions, explained in section III-A1. As a basis for our analysis, we mainly considered the synthesized results described in subsection III-A5. In the following subsections, we present detailed descriptions, comparisons, analysis, and interpretations of the findings. To maintain adequate focus and flow, the discussions are ordered according to the research questions. The results revealed that multi-agent IDS architectures have three classifications: organizational structures, agent types, and computational components classification. The data related to this research question is shown in the appendices A, B, and C. In the following subsections, we will give detailed explanations of each classification.

1) THE CLASSIFICATION OF ORGANIZATIONAL STRUCTURES
Organizational structures provide frameworks for agents' interactions through the definition of roles, behaviour expectations, and authority relations. They also impose constraints on the ways agents communicate and coordinate [22]. The results uncovered that multi-agent IDS architectures constitute three organizational structures. First, the hierarchical structure that is the most common among the current multi-agent IDS architectures, it was adopted by 27 studies, see Appendix A. In this structure, agents are assigned specific tasks and distributed over the network in a hierarchical manner, and data flows from multiple sensors, located at the bottom of the hierarchy, to one control agent at the top. The main issue of this structure is the central point of failure.
Second, a decentralized (or distributed) organizational structure is also found. In this structure, the architecture agents organized in small groups belonging to the same subnet or network segment. The data flow from multiple sensors to one control agent residing in the same network segment. In some studies, the analysis agents are organized in a layered style, which resembles the hierarchical model, and the results of lower analysis agents fed to the analysis agents of the next layer until it reaches to the security centre on the top of the hierarchy [19]. This structure was adopted by 24 studies, see Appendix A. The main issue of this structure is its complexity Third, a dynamically adaptive structure was also found. In this structure, the architecture agents change their behaviour dynamically to adapt to network changes [5], [35], [37]. The dynamic behaviour of this structure is usually implemented using mobile agents or by removing and instantiating agents immediately when changes happen in a network. The agents of this structure incorporate adaptation techniques to enable them to respond to environmental changes. This structure was adopted by 20 studies. The dynamic adaptive structure is the most appropriate structure for open and dynamic environments. The disadvantage of such a structure is its complexity.
Additionally, the considerable development of computer network technologies resulted in very different network types such as LAN, WAN, MAN, VPN, Adhoc, MANET, and others. In this study, we consider only the main two categories of computer networks, wired and wireless. Generally, the organizational structures of multi-agent IDS Depend on the computing environment they deal with, for example, the wireless network is dynamic and requires a dynamically adaptive structure to face the dynamism of such networks. Figure 4a shows the classification of the multi-agent IDS architectures based on their organizational structure.

2) THE CLASSIFICATION OF GENTS' TYPES
Agent types represent the features that describe the internal capabilities and functionalities agents have [39]. The results showed that the existing multi-agent IDS architectures encompass several agent types, including autonomous, intelligent, adaptive, reactive, proactive, cooperative, collaborative, and deliberative agents. Almost all the studies under investigation adopted autonomous agents in their approaches. The autonomous agents have the ability to work and maintained by their own [40]. All the proposed architectures adopted intelligent agents by incorporating Artificial Intelligence(AI) techniques like machine learning, soft computing, and immune system mechanisms. Several of the proposed approaches adopted adaptive agents that can change their behaviour according to the environment changes. Also, they can adjust their abilities depending on the parameters they received from the other interacting agents [20]. Also, the results show that the current architecture incorporates reactive and proactive agents. The agents that do misuse detection are considered proactive, while the agents that do anomaly detection are considered as reactive agents. The reactive agents monitor their environment and react to the changes that occur in a timely fashion. Reactivity of an agent also means the ability to immediately adjust its behaviour when the environment situations change. A proactive agent must show opportunistic behaviour and take the initiative at the right time. Most of the studies adopted cooperative and collaborative agents in their architecture. Cooperative agents have different beliefs and reasoning methods, and they share a common goal, while collaborative agents share a common objective but keep their individual goals [41]. Few of the selected studies adopted deliberative agents in their architectures [43]- [45]. This type of agent can reason using built-in knowledge. There is also competitive agents used by [49]. Figure 4b illustrates the classification of multi-agent IDS architectures based on agents' types. For more details on the types of agents used, see Appendix B.

3) THE COMPUTATIONAL COMPONENTS CLASSIFICATION
The computational components refer to the algorithmic techniques that the agents include to achieve attack detection tasks. In this SLR, we found four computational components in the proposed architectures, namely data collection and synchronization, data analysis, management and coordination, and knowledge sharing. First, data collection and synchronization in the current multi-agent IDS architectures is a distributed process that involves multiple sensor agents located in multiple places on the network to collect, aggregate, and prepare data for the analysis process. In the hierarchical and decentralized organizational structures, the data produced by sensor agents at lower layers travel upwards through the hierarchy to upper layers to provide a broader view about current incidents. On the other hand, in the dynamically adaptive structure, this process is quite challenging as agents change their locations and behaviours. Multi-agent data collection has not sufficiently addressed in the literature yet. However, there are few studies proposed techniques for data merging and synchronization using time interval [10], [45]- [47], IP address [48] and attack type [42], [49], [50].
Second, data analysis is the process of manipulating the collected attacks data by analyzer agents to detect incidents. The analyzer agents, can be misused [47], [51], [55], anomaly [46], [52], [53] or mixture of both (hybrid) [9], [48], [54]. The pre-process and analyze the data using technologies such as statistical methods and AI methods such as machine learning, soft computing, and biologically inspired methods. The analysis process can be done by an individual agent or collectively by a team of agents. The location of the analyzer agent can be either centralized at a specific location in the network, such as the security centre [19], [20], [55], or decentralized at several points on the network [45], [56], [57]. Fig. 5 illustrates the classification of the analysis techniques used with multi-agent IDS architectures.
The third component is the management and coordination component that used to configure, organize, and maintain the multi-agent IDS architecture. In some cases, this component is manually managed by an administrator [6], [11], [53], while, in other cases, it is self-managed [37]. In the dynamically adaptive structure, this component is responsible for all self-management tasks. Finally, the knowledge sharing component is used for communicating data and results among the agents. All the architectures' agents use knowledge sharing to inform the other agents by their actions. There are three methods of knowledge sharing: a shared knowledge-base or ontology [4], [58], a distributed knowledge-base [47], [59], [60] and message exchange scheme [14], [19], [61]. In some of the selected studies, the proposed architectures exploited mobile agents to exchange knowledge among agents [62].
From the discussion of this research question, three classifications of the existing multi-agent IDS have been found. The first classification categorized the multi-agent IDS architectures based on their organizational structure, as illustrated in figure 4a. The second classification categorized the multi-agent IDS architectures according to the agents' types, as illustrated in Figure 4b. The third classification is based on the techniques used for data analysis, as illustrated in 5. Also, detailed information is shown in Appendix A. We identify that these properties are vital features that influence all other aspects of the architectures. More specifically, these key features have impacts on task distribution, data collection and synchronization, management and coordination, and knowledge sharing. Furthermore, these key features also influence the characteristics of multi-agent IDS architectures such as learning and adaptation, communication, scalability, and reliability. In the following subsections, we will discuss in detail how aspects such as task distribution, data collection and synchronization, management and coordination, and knowledge sharing can affect the performance of multi-agent IDS architectures and their characteristics.

B. RESEARCH QUESTION RQ 2 (WHAT ASPECTS OF MULTI-AGENT IDS ARCHITECTURES INFLUENCE INTRUSION DETECTION PERFORMANCE?)
For answering this research question, we consider the previous discussion on question RQ 1. In the light of what has been discussed there, it can be concluded that there exist interrelationships among the organizational structures, the agents' types, and the computational components of multi-agent IDS architectures.

1) AGENTS AND TASKS DISTRIBUTION
Tasks distribution is the process of decomposition and distribution of problem-solving tasks among multiple agents [41]. In multi-agent IDS architectures, agents are allocated subtasks and distributed according to the chosen organizational structure, which determines how sensor agents will be placed, whether there will be individual analyzer agent or multiple analyzers, and how the architectural management processes will be dealt with. With respect to sensor agents, it was observed all the organizational structures had adopted distributed sensor agents to capture attacks (e.g., distributed and coordinated attacks) traffic from the network [63]. The distribution of multiple sensors can increase the system scalability by increasing the number of agents to collect and pre-process large volumes of data concurrently.
Second, as for analyzer agents, the multi-agent IDS architectures exhibited two ways, individual and multiple analyzers. Using multiple analyzers will help in load balancing by dividing the workload among multiple analyzer agents for parallel execution [9]. There is also another benefit of using multiple analyzer agents, which is the creation of multiple agents with different analysis techniques to analyze sophisticated types of attacks that cannot be detected by a single analysis technique. That is because, in some cases, specific techniques fail in analyzing specific types of attack data, while others can analyze them effectively [4].
Third, in most of the proposed approaches, the management task of the multi-agent IDS architecture is centralized and manually achieved by an administrator who interacts with the system through a user interface (console) to accomplish the management tasks such as creating, adding, or deleting agents [11], [19].

2) DATA COLLECTION AND SYNCHRONIZATION
Based upon our studies, the existing multi-agent IDS architectures disclosed that data collection and synchronization had not been discussed sufficiently in the literature. However, there are a few studies that also included tasks such as data collection, aggregation, synchronization, and preparation for analysis by analyzer agent(s). These tasks, in some cases, were embedded in sensor agents [58], [64], while in other cases, they were added to the tasks of analysis agents [57], [65]. As apart of data collection and synchronization, sensor and analysis agents also encompassed methods to generate and derive new features, from the accumulated data, these features assumed to be effective in classifying the attack incidents [4], [5], [54].
One of the issues that challenge efficient data aggregation is to merge data from different sources. This problem has not been discussed sufficiently in the literature, though some studies contained schemes for merging and synchronizing data by source and destination IP addresses [46], [48], timestamps [10], [45]- [47], and protocol type [42], [49], [50].
The data collection and synchronization processes affect detection performance in different ways. One way is that the complexity of aggregation methods may degrade the detection performance by increasing the processing cost. Additionally, if there is a separate agent for data aggregation, this can also add additional communication overhead on the system as the agents need to communicate. Consequently, the throughput of the architecture will be reduced, and the response time would be increased in contrast. Furthermore, data aggregation methods also affect the detection accuracy by the quality level of the generated features.

3) DATA ANALYSIS
The data analysis component of multi-agent IDS architectures is the most crucial component because it carries out data processing and analysis. The agents learn while they VOLUME 8, 2020 are analyzing data using techniques considered the core of multi-agent intelligence. The analysis techniques used with the current architectures can be divided into four categories: statistical methods, AI, soft computing, and immune system techniques, Appendix C shows the analysis methods used with the current architectures.
Our investigation disclosed that most multi-agent IDS architectures actually use single-agent learning, and there is no clear definition of team learning in multi-agent IDS architectures except in few studies that used some sort of multiagent learning but not exactly team learning, such as [16], [66]. Multi-agent team learning is a very complicated task, but it has great benefits on the agent's rationality [22] and concurrent learning [41].
The techniques used with the data analysis component may affect intrusions and attacks detection performance in two ways. First, the processing cost of some techniques, such as ANN, is very high, and this could cause a delay in response time. The computational cost of data analysis techniques of multi-agent IDS architectures is liable to the analysis technique complexity, and also the data amount needs to be analyzed. Complex AI techniques used to consume too much CPU time and RAM space rather than simple methods such as statistics.
Second, the analysis component is also affected by the selected features and the effectiveness of the analysis technique chosen in data classification. An example of this, some features are useful in detecting some attack types; while they are not in detecting other attack types. Also, there is some classification technique that is effective in classifying some attack types; but they are less effective in other cases.
In regards to network performance, team learning agents can use distributed data analysis and provide a mechanism for load balance. In the case when massive attacks such as coordinated DDoS and worms strike a network, the data volumes can suddenly become very big for IDS to process in real-time. To solve this situation, multiple analysis agents can divide the workload among them and process the data concurrently. There are several studies that use multiple analyzer agents with multiple analysis techniques to balance the load and benefit from multiple analyzers [11], [67], [68]. The grouping of the data analysis methods and their advantages and disadvantages are explained in Appendix C.

4) MANAGEMENT AND COORDINATION
The results showed that the existing multi-agent IDS architectures achieve coordination and management tasks by using a separate agent called manager, coordinator, or moderator, such that the proposed architectures in [14], [20], [61]. The manager agent performs management tasks either manually or automatically by using a self-management mechanism. In the case of manual management, an administrator is in charge of performing all the management operations such as adding, removing, and configuring architecture agents [11], [19]. The disadvantage of this method is that the IDS architecture is completely un-configurable and un-scalable without an administrator. Therefore, the system cannot change, adapt, or extend by itself to face the environmental changes. In the automatic or self-management architecture, there is no administrator, and the IDS architecture can automatically adapt to environmental changes. This needs intelligent mechanisms to check out environmental changes and react to them by adding or removing agents. Quite a few architectures use the automatic management scheme [37].

5) KNOWLEDGE SHARING
Knowledge sharing is an essential part of multi-agent intelligence. This SLR found out that there are three different knowledge sharing schemes used with the existing multiagent IDS architectures. First, some architectures adopted shared knowledgebase or, in some cases, shared ontologies. A shared knowledge base or ontology represents a central hub for all agents to exchange their desires and beliefs; this scheme adopted by [4], [9], [15], [58], [59]. The problem of this scheme is that the multi-agent IDS architectures become susceptible to the risk of central point of failure, central the hub may face errors and crashes. Also, this scheme adds extra communication overhead because the architectures agents contend at the shared point to communicate their pieces of knowledge. Moreover, the processing time of the communication messages will increase the overall time of data analysis.
Second, some of the existing architectures adopted distributed knowledgebase schemes. In these schemes, every agent has its own knowledge base, and it needs to synchronize it with other architecture agents [47], [59], [70]. This also adds additional communication overhead to the system architecture because agents need to communicate with each other to synchronize their knowledgebase.
Third, all the existing architectures used message exchange schemes. Such schemes are considered vital for architectures' agents to cooperate and collaborate to achieve their goals. The main disadvantage of this scheme is that there could be extreme communication overhead among the agents if they use an inefficient cooperation protocol (e.g., unconstrained interactions among agents).
From the discussion of this research question, we conclude that there are very strong interrelationships among the properties and characteristics of multi-agent IDS architectures and the performance of attack detection. The discussion focused mainly on the effects of agent distribution, data collection and aggregation, data analysis, management and coordination, and knowledge sharing. We found that the mechanisms and techniques used have direct effects on the performance of attack detection. Based on what has been discussed, we can say that the more intelligent the mechanisms and techniques used, with multi-agent IDS architectures, the faster and accurate it is attack detection. Figure 6 summarizes the interrelationships among multi-agent IDS architectures properties, characteristics, and intelligence.  The discussion of question RQ 2 emphasizes that the properties and characteristics of multi-agent IDS architectures influence the overall performance of intrusions and attacks detection. The discussion also confirms that although multiagent IDS architectures have advantageous characteristics, but they also suffer limitations that can eventually degrade the overall performance of intrusions and attacks detection. In the following subsections, we will make detailed discussions on the limitations found in the existing multi-agent IDS architectures. The discussion will focus mainly on the limitations that may impact the response time and accuracy of intrusions and attacks detection.

1) SCALABILITY
Scalability means the ability of an application to grow up to meet the increasing performance demands [71]. In other words, it means the ability to incrementally add agents to a system for processing volumes of data [41]. In this SLR, the scalability of multi-agent IDS architectures will be discussed from an architectural perspective where the system automatically scales up by instantiating agents (e.g., sensors and analyzers) as needed. On the other side, as the number of agents grows, the state space will grow as well.
Multi-agent systems are inherently scalable; this is so because of the innate modularity they are supposed to have. So, the techniques used for handling this modularity are of importance [39]. Based on the obtained results, the scalability of multi-agent IDS architectures was not sufficiently addressed. However, there are some architectures that provide untested mechanisms and techniques for scalability. All three organizational structures (hierarchical, decentralized, and dynamically adaptive) suffer from limitations in scalability.
In the hierarchical structure, there are few architectures that have mechanisms to provide scalable IDS. In [20], the proposed architecture included a method that creates agents with specific tasks and distributes them across the layers of the hierarchy to compose the IDS. In this study, the degree of scalability is still not sufficient as the new agents are located in the same host and share the same system resources. The proposed architectures in the studies [9], [11], [72], [74], provide mechanisms for scalability by instantiating new agents when needed. These architectures didn't enforce any constraint on agents' creation. Thus, creating agents without restrictions will exhaust computer system resources and degrades the performance of intrusions detection.
The decentralized structure includes only two study that deals with scalability. In [48], the analysis agent used to analyze data and replicate itself as needed. The drawback of this algorithm is that this mechanism dealt with scalability in the same host and didn't deal with the network factor. Additionally, it does not consider the limitation of host resources. The proposed architecture in [14] manage scalability by dividing the network traffic to subparts and process these subparts simultaneously using a group of agents distributed over the network. In [66],task decomposition and team learning were used, that instead of having one big DDoS attack problem at the victim machine, there would be multiple smaller DDoS at the team leaders.
In the dynamically adaptive structure, there are also few studies dealt with scalability. The proposed architectures in [35], [61] used simple methods for adding and removing agents. These methods didn't enforce any constraint on agents' creation and deletion. In [62], a technique to transfer data to the neighbouring nodes using mobile agents was used. This technique adds additional communication overhead to the system because the mobile agent repeatedly travels among hosts carrying data. In addition to that, mobile agents have security issues reported in [75]. In [59], the proposed architecture was designed to scale up dynamically by creating a random number of sensor and analyzer agents to gather and analyze data related to attacks. The random creation of agents exhausts system resources. For this reason, there should be limits and constraints for creating agents. In [60], [76], [102], the proposed architectures use mobile agents to search for the most effective nodes to analyze data, but again, the use of a mobile agent will add communication overhead.
The proposed architecture in [54] used a mechanism that selects the best analysis agents, in terms of their clustering capabilities, to replicate them for analyzing data. The disadvantage of this mechanism is that it does not enforce strict constraints on agents' replication. The architecture should be expanded only when exposed to massive attacks, such as worms and DDoS; otherwise, it will be a consumption of system resources. Agents' creation algorithms should take into consideration system and network resources constraints.
The architectures proposed in [60], [76], scale up by moving agents to the nodes with useful classification capabilities.
The disadvantages of this approach are the use of mobile agents that suffer from security issues reported in [75].
From this discussion, we conclude that the scalability of the multi-agent IDS architectures suffers limitations in the methods that have been proposed so far. Therefore, it is necessary to develop and evaluate models, frameworks, and approaches to provide scalability as an essential characteristic in the architectures.

2) LOAD BALANCING
Load balancing is the distribution of workload among multiple agents to optimize resource usage, maximize throughput, minimize response time, or to avoid overload on a single resource [77]. This characteristic is interrelated to scalability; a scalable system is a load balancing necessarily. That is because when tasks are divided and distributed among architecture agents, the workload also has to be divided. The obtained results reveal that the load balancing of multi-agent IDS architectures was not sufficiently addressed. But, there are some architectures that provide untested mechanisms and techniques to handle situations when IDSs congest and need for loads balancing. Based on the selected studies, all three organizational structures suffer from load balancing limitations.
In the hierarchical structure, there are some architectures used mechanisms for load balancing. In [20], the proposed architecture encompassed an algorithm for dividing the workload among various agents and avoid the centralization of traffic. The algorithm also can change agents' roles depending on the current network status. There are no details on how the architecture scales up to accommodate data when attacks are launched. For instance, when massive attacks, such as DDoS and worms, occur in a network, the algorithm doesn't show how the workload is divided and distributed among agents to process the huge attack data. The method should take into account the amount of data expected to be processed so that the IDS estimate the required number of analysis agents. In [63], the proposed architecture included a cooperation mechanism that distributes the workloads among agents in the neighbouring hosts. The architecture included a global intrusion detection agent to communicate with its neighbouring agents to detect attacks or to use storage. In this architecture, the total number of agents remains the same, but the load is distributed among the neighbouring agents. The issue in this mechanism is that it does not handle the situations when all agents are loaded. In [7], the proposed architecture used an algorithm for distributing the detectors (attack data) across hosts to improve the processing time. This method distributes the workloads arbitrarily across the network without constraints to choose which host is the best choice for processing the data. Because if a host is already loaded, adding more jobs will decrease throughput. In [78], the proposed architecture provides minimum network load and better CPU utilization by shadowing log file. Shadowing log file is considered as work duplicated, and this also increases the processing time. The proposed architectures in the studies [9], [11], [72], [74] included mechanisms for distributing the classification tasks throughout the various layers of the architectures. The mechanisms dealt with distributing the tasks but didn't handle how a workload is divided among the architecture agents.
In the decentralized structure, there are a few studies that used mechanisms for load balancing. In [101], A method to control the CPU usage ratio of the IDS server with both normal traffic and flooding attacks was used. The method works by managing registration requests of the clients to keep low CPU load on the IDS server. The proposed architecture in [79] used a mechanism for applying a filtering operation that matches between the captured traffic and the intrusion database to ensure that only the unclassified traffic will be processed.
In the dynamically adaptive structure, also there are a few studies used mechanisms for load balancing. In [14], [61], the proposed architectures balance the load by creating multiple analysis agents, based on the network traffic and the availability of logical processors. Then, the architecture divides the dataset into sub-datasets. This method does not enforce any rules on the agents' creation. An algorithm based on a dynamic election idea, to search and choose agents with fewer loads for analyzing attacks' data, was proposed in [48]. The election algorithm limits search operations (for agents) to be in the same network segment and didn't take into account the other segments. The issue is that if the same network segment is loaded, there should be a chance to move the analysis processes to other segments with less traffic. Another election based mechanism was proposed by [5]. The mechanism was used for searching and choosing the agent with the lightest load to replace the current agent, which is having a higher load due to an attack. The algorithm searches the whole agents' list, using a binary search or group search until it finds a management agent with the lowest load. Although this algorithm provides load balancing, when attacks strike, such as DDoS or Worm attack, the whole network is loaded, not only the analysis agents. For this reason, the constraints should include other factors such as host and network states.
The previous discussion shows that the load balancing mechanisms used with multi-agent IDS architectures suffer limitations. So, it is necessary to conduct experiments and propose models, frameworks, and metrics to create and evaluate this characteristic on multi-agent IDS architectures.

3) LIMITATIONS OF RELIABILITY, FAULT TOLERANCE AND AVAILABILITY
Reliability, fault tolerance, and availability are interleaved characteristics defined, in an article published by Microsoft [71], as follows: 1) the reliability is the probability of failure for a single solution component. The system is reliable if it is fault-tolerant; otherwise, it might face unavailability. 2) fault tolerance is the possibility of a system architecture to continue functioning when parts of it face failure. 3) availability means the percentage of time that a system can achieve its intended functions.
Based on the obtained results, most of the hierarchical architectures do not have these essential characteristics because there is always the issue of a single point of failure. However, some of the existing hierarchical architectures used some mechanisms to recover from errors and failures. For making the system reliable, an auto fault tolerance characteristic was developed by [78] using mobile agents. In [9], the proposed architecture comprised a mechanism to create new agents to recover from errors and problems. This study doesn't provide any details on the performance of the mechanism. In [63], the proposed architecture used a reliability technique to make each of the architecture's components monitored by a different component. This method is not a useful solution for providing a reliable operation on the architecture. The communication overhead among the components can cause high traffic on the network, which can lead to undesirable results such as delay in response. In both [7], [73], the robustness of the proposed architectures comes from integrating several detecter agents with several classification algorithms and manage them to detect attack incidents.
There are few studies related to the decentralized and distributed structure that provides these characteristics. In [44], [45], the architectures used an algorithm based on Byzantine Agreement Protocol (BAP) to detect and isolate the compromised node. This is still not enough for the architecture reliability as the mechanism doesn't include a recovery technique to restore the system architectures. In [79], the proposed system used an algorithm that stores each piece of data in three locations to promote fault tolerance and recovery.
The dynamically adaptive structure has a few studies showing these characteristics. The architecture in [5] used a dynamic selection algorithm based on an improved Bully algorithm for timely system recovery when an error occurs due to the higher load at the management agent. The algorithm focused on the failures due to the higher load and didn't address the problems when the agent is dead due to network and resource congestion. In [59], the architectures are fault-tolerant by using self-diagnosis, self-healing, and selftesting mechanisms to detect abnormal behaviour, recover and repair the damage. A robust communications protocol for multi-agent IDS was proposed to handle transmission losses [60], [76]. This protocol didn't handle agents' failures or system crashes.
From the discussion, the existing multi-agent IDS architectures suffer limitations in reliability, fault tolerance, and availability. Therefore, it is necessary to propose and evaluate robust and fault tolerance methods on multi-agent IDS architectures.

4) ADAPTATION AND LEARNING
Adaptation means the ability of an interactive system to change its behaviour according to environment changes and information acquired from the surrounding agents [80], [81]. The term learning refers to the process that includes all computations such as knowledge sharing, knowledge acquisition, classification, prediction, inference, and decision making that are executed to achieve a particular learning goal [82]. Adaptation, learning, self-organization, and self-configuration are interconnected characteristics. These characteristics are significant for what is called self-management [83]. Selforganization is the ability of a system to automatically arrange its components and elements in a suitable way without any external help [69]. Self-configuration is the ability of a system to automatically control changes in its parameters to produce the desired output [84].
The results reveal that the adaptation of multi-agent IDS architectures is twofold, adjusting agents' states to respond to new attacks or changing system parameters(e.g., traffic, agents number) to respond to environmental changes. Adjusting agents' internal states is the connection point between adaptation and learning. With respect to this, the results uncovered that there are several adaptation mechanisms used for changing agents' internal states to enable them to detect newly seen attacks. Generally, these mechanisms rely on continuously updating agents' knowledgebase with the help of machine learning, soft computing, and other AI techniques. The suggested architecture in [6], used a simple mechanism, based on association rules, that changes the profiles of the typical behaviours to enable the analysis agents to detect the new abnormal behaviour. The main issue of this mechanism is that creating rules needs prior knowledge, which will not be available most of the time. Another issue of interest is the use of single-agent learning and not using cooperative learning; when a network experiences a DDoS attack, for example, agents' cooperation is required to collect and jointly process the attack data for fast and effective detection.
The proposed architecture in [46], a rule-based algorithm using information theory, was suggested to detect when abnormal behaviour is detected; the system enters a selfdiagnosis mode to categorize the fault and get detailed information about the incident such as source IP address and symptoms on the system. Once these details are identified, the system adapts by generating features to be tested to find the best ones for building classification rules. This mechanism uses labelled data, which will not be available in all cases.
The adaptive architecture proposed in [45] uses a Bayesian-based learning algorithm to enable the agents to detect new attack types. In addition to the prior knowledge required by the Bayesian algorithm, also the manual confirmation by the system administrator is needed, which is impractical.
The proposed architecture in [54], uses an evolutionary algorithm in association with a Naïve Bayesian classifier to estimates the probability of intrusions' activities. The algorithm used genetic algorithm to let agents continuously learn from the environment. Also, in [85], an evolutionary algorithm was used in combination with game theory. The computational cost of evolutionary algorithms is very high esspecialy in processing huge data. The same problem exists in [86], [94] where a bunch of classification algorithms used VOLUME 8, 2020 for evaluating the performance of multi-agent architecture. In [87], a knowledge-base with a reasoning algorithm is used, but still, the problem of learning needs to be fixed, because the prior knowledge is not available in the case of new attacks.
The proposed architecture in [67], uses a technique composed of a series of machine learning techniques include Online Random Tree, Online Random Forest, Online LaRank, Online Multi-Class Linear Programming Boost, and Multi-Class Gradient Boost. These techniques fused with multi-agent to detect malicious behaviours. The main issue in this approach is the blending of human expertise with modern artificial intelligence. Human expertise is difficult to extract and also takes time to be translated into rules.
The architectures in [7], [64], use mechanisms inspired by the immune system and genetic algorithm to generate a knowledge base of attacks to immediately respond to the attacks in the future. The algorithms have two issues: limitations in learning and high computational cost. The learning limitation is that the algorithms need to learn from previously labelled data, which will not be available in the shadow of the very rapid growth of Cybersecurity attacks. The high computational cost of these mechanisms is caused when matching the suspicious network connections with the whole self and non-self-detectors lists. This causes computational overhead that consumes system resources (CPU time and RAM) and eventually will cause a delay in response time. Also, Genetic algorithms do not scale well with complexity [88].
In [52], the proposed architecture contains an immune system based algorithm that selects the best analyzer agents to randomly replicate them to classify attacks, based on a fitness function with parameters such as number of agents, number of classification records etc. The issue of this algorithm is that the fitness function does not consider any of the environmental factors such as system resources or network resources. The mechanism might fail in case the specified agent resides in a very busy host or network segment. On the other hand, this mechanism clone agent randomly, which is considered as resource consumption.
The architectures in [59], [70] include learning and adaptation mechanisms based on the immune system to enable agents to detect new abnormal activities. The problem of this mechanism is that the agents need labelled data for training, which is not always available. Another issue in this mechanism is that the adaptation mechanisms did not take into consideration how the architecture agents will adapt to the environmental changes. For example, when a network is exposed to high traffic because of an attack, the architecture agents could immediately adapt by increasing the number of analyzer agents, choosing analysis techniques, or isolating the suspicious nodes to reduce the attack invasion. The suggested mechanism doesn't show how the agents get involved in cooperative learning.
The architectures proposed in [42], [89], [90] use an adaptation mechanism inspired by the immune system to adjust the architectures' agents according to environmental changes. The mechanisms adapt in three stages: the first stage is called diversity generation, which is the generation of different agents with distinct specificity by using mutations. The second stage is called self-maintenance, where the agents are adjusted to be insensitive to known attacks during the development phase. Third, is the phase called the memory of non-self, where agents are adjusted to be more sensitive to unknown attacks. The issue in this mechanism is the excessive creation of the agents that will consume system resources. On the other hand, making the agents sensitive and non-sensitive is a recurrent process based on calculating the danger degree using mathematics, and this will also cause high computational cost. The danger theory algorithm of the immune system also utilzed by [99].
The architecture proposed in [48] includes an adaptation mechanism based on an improved Artificial Neural Network(ANN) algorithm that adjusts the learning rates adaptively. The performance of ANN is exceptional when a huge number of training data is used. However, one of the ANN problems is called over-fitting, which occurs when the network stores all training examples, but it is unable to generalize to new cases. From another side, the huge data volumes cause high processing cost.
The proposed architecture in [91], uses an adaptive adjustment sub-system, based on ANN, to automatically adjust the system to detect new attacks, by using the information provided by the environment, or manually by an administrator. The problem of this mechanism is that it requires manual intervention from an administrator.
The proposed architecture in [5], uses a mixture of the anomaly and misuse techniques. The anomaly technique implemented to improve the misuse detection applying correlation analysis, sequence analysis, cluster analysis, classification analysis, rough set analysis, and outlier analysis to update rules library and eliminate the old rules set.
In [38], the proposed architecture uses a Qualia based principle that takes the analysis results to modify world models to use in the future. The architecture also uses prior knowledge and agents' own experience in making decisions.
The architectures proposed in [16]- [18], [92], [105] encompass reinforcement learning algorithms with fuzzy logic (as a function approximation) to adapt by selecting the best strategy for detecting attacks and responding to it. The main issue in these architectures, as stated by the researchers, is that convergence may not occur, and that means the optimal solution is not guaranteed. The same problem persists in the reinforcement learning mechanisims proposed in [66], [103], [104].
A trust based adaptation technique integrated with a distributed agent-based architecture for detecting DDoS attacks in WLAN was proposed in [93]. The technique was used for detecting and isolating the attacks. A trust mechanism is a rating process between two peers based on their historical performance. The problem is that if no historical, the agents will not be able to detect the attack.
In the proposed architecture in [38], the agents use a learning model that extracts new information from the surrounding entities and environment to adapt itself to new threats with little or no human intervention.
In [15], a self-learning ontology was proposed using Intuitionistic Fuzzy Logic (IFL) to generate new attack rules. The problem in this mechanism is the need for labeled data, which will not be available in the case of the swift evolution of cyberattacks.
The suggested architecture in [62], comprises an algorithm based on Support Vector Machine (SVM) for learning and prediction of new attack types. The algorithm trains the analyzer agent directly by using a dataset to create a model and then uses that model for classifying the new attack types. The problem of this algorithm is that it doesn't deal with how the architecture agents could learn from each other or from the environment. SVM was combined with extreme machine learning (ELM) technique and used by the adaptive architecture proposed in [35].
With regards to the adaptation mechanisms that were used with multi-agent IDS architectures to change agents' behaviours to respond to environmental changes, these mechanisms involve tasks such as adding, removing, or changing agents' goals. The proposed system architecture in [62], adapts to the environment by using a mechanism based on mobile agents. There are critical issues for mobile agents mentioned in [75]. Thus, mobile agents are considered a drawback for this mechanism.
The adaptive architecture in [53] comprises a mechanism that dynamically adapts to environmental changes and attacks. Based on a condition, intrusion detection will be achieved by the basic agent, local coordination agent, or global coordination agent. The disadvantage of this process is the long steps to follow in case an event could be detected neither by the basic agent nor by the local coordinator agent.
In [54], the proposed architecture comprises a mechanism based on a genetic algorithm for adding and removing clustering agents according to their fitness. The fitness is a value calculated to determine an agent's ability to cluster data. The clustering agent that produces clusters with high dispersion is considered less effective than the clustering agent that produces clusters with low dispersion of elements. Therefore, the later is replicated, and the former is removed. For replicating the clustering agents, the algorithm considers only the internal agent's state (clustering ability) and ignores the other factors such as system and network status. Suppose the best agents reside in a very busy area in the network, using this adaptation algorithm will increase the response time due to the increased processing cost.
In [76], [102], the proposed architectures include a reputation based algorithm to dynamically find nodes with a high ability to classify network activities, and a multi-objective evolutionary algorithm is enforced to help agents search for useful operational parameter values for classification. The reputation algorithm is used for instructing agents to migrate to other nodes or to share information with other agents. One of the disadvantages of this approach is the security issue related to the mobile agent mentioned in [75]. On the other hand, the central agent controller is liable to the risk of the central point of failure.
The suggested architecture in [67] includes an adaptation mechanism that has two detection engines, misuse, and machine learning. When a malicious multi-agent system changes its behaviour to evade detection, the misuse engine gradually stops warning of the current malicious traffic, and the machine learning agents continue to investigate and alert of the new behaviour. The architecture also uses an election algorithm to choose between multiple machine learning techniques to analyze network traffic. The problem in this architecture it that the supervised machine learning techniques need labelled data which will not be available most of the time in the shadow of the very fast evolution of attacks.
In [5], when a network experiences heavy load because of an attack, one of the management agents initiates an election process by communicating with the other management agents in the network to examine their loads. For each management agent, if its load is less than the load of the management agent that has initiated the election process, the agent will reply by a positive election result. Then, the initiator agent selects the agent with the smallest load to start the analysis. If the initiator agent receives no result, then it will replicate itself and start the analysis process. Although the empirical evaluation of this study was not presented, it can be noticed that the proposed mechanism considered only the load of the agents, and didn't consider the other factors that might affect the detection performance, such as the availability of system resources. In the architecture proposed by [95], an adaptation scheme depends on attack severity was used. This scheme utilized a paramater called relationship metric that characterizes the distribution of clients. A too high value of this metric is considered as abnormality.
The proposed architecture in [19] includes an adaptation mechanism inspired by the biological immune system to automatically create two types of agents: an intermediary agent with a memory of detectors (attack data records) and a superior agent, which is a mobile agent. The intermediary agent is used for identifying attacks, and the superior agent is used for moving across the network to perform autodestructive processes. The use of mobile agent adds additional load on the network because it travels from host to host to destruct attacks and their consequences. Furthermore, the sequential movement of the mobile agent, from host to host to undo the damage caused by attacks, can cause a delay in response time.
In [9], [11], [43], [72]- [74], the architectures use mechanisms that consist of two types of intelligent Case-Based Reasoning(CBR) agents used to learn and adapt to changes in attack patterns and user behaviour. The adaptation mechanisms divide the classification task into two phases. In the first phase, a process called initial filter is implemented to detect simple attacks without using a large number of system resources. In the second phase, more complex computations are performed, and that requires a large amount of computer system resources. The mechanisms can adapt by enforcing a load balance mechanism to save system resources. A mechanism based on CBR agents was also proposed by [20] to reuse past knowledge to solve new problems.
With respect to the relation between adaptation and selfmanagement, the suggested self-managed architecture in [37] includes a mechanism to dynamically adapt to environmental changes, tune resources, discover, diagnose, react to disruptions, and anticipate detection, identification, and protection against threats. This mechanism uses an autonomous central agent that can diagnose failures, and manage the situations when higher loads and communication overhead arise in the system. There are no enough details on the self-management mechanism; however, the autonomous central management agent is liable to failures.
From the previous discussion, the existing multi-agent IDS architectures have limitations in the adaptation mechanisms, and it is required to conduct experiments on more enhanced adaptation models and frameworks. Also, there is a need for proposing standard measurements and metrics for evaluating the adaptation of multi-agent IDS architectures.

5) MULTI-AGENT LEARNING
Learning is an essential part of multi-agent intelligence. It comprises two types: Single Agent Learning(SAL), which means how an individual agent improves its learning abilities, and Multi-Agent Learning (MAL), which means how a group of agents cooperate in analyzing data and learning effectively in a multi-agent environment [82]. Learning techniques are always embedded in the data analysis components of multiagent IDS architectures.
Our investigations on the existing multi-agent IDS architectures manifest that true MAL has not been achieved yet. There are several important aspects of multi-agent learning that have not been covered until now in the literature of multiagent IDS, for instance, distributed AI, parallelism, interactions, and learning methods. So, in this subsection, we will limit the discussion on the few multi-agent IDS architectures that exhibiting some characteristics of MAL.
The multi-agent IDS architectures that adopt the approach of multi-agent learning; they use different mechanisms to let the agents communicate with each other to improve their knowledge. The proposed hierarchical architectures in [16], [18], adjust their learning parameters through fuzzy Q-learning to detect future attacks. The architectures' agents cooperatively learn to adjust their parameters a mechanism based on game theory. The proposed game theory approach was limited to only two agents, so the true MAL doesn't exist in this architecture.
In the proposed architecture by [17], a cooperative fuzzy artificial immune system mechanism was proposed to improve the agents' self-learning capacities and provide the agents with an incentive function to protect the most vulnerable sensor nodes. There are two issues in the mentioned architectures: first, using Fuzzy logic need human effort for designing fuzzy rules, which is not practical while there exists a huge number of new attack every day. Second, the cooperation using the hierarchical structure increases the communication overhead as the number of the hierarchy layers' increases.
From the previous discussion, the learning of the multiagent IDS architectures has two aspects: 1) individual agent learning, 2) cooperative multi-agent learning. From the selected studies, the learning mechanisms that used with the current multi-agent IDS architectures have limitations that can be concluded in: incremental agent learning, agents learning from the environment, and cooperative multi-agent learning. The proposed approaches also ignored how agents infer knowledge by their own(reasoning). For this reason, the multi-agent IDS architectures need more improved frameworks, models, and algorithms for enhancing learning capacities.

6) COMMUNICATION OVERHEAD
In a multi-agent system, the communications related to negotiations or the transfer of high volumes of information causes significant overhead that leads to delay in the systems with strict time and bandwidth limits [96]. Real-time applications such as IDSs should strictly enforce rules on agents' communications to prevent system bottlenecks and delays that lead to low throughput [41]. Multi-agent IDS architectures typically fall into this type of system, because they need to transfer huge data in real-time. From the selected studies, it was observed that communication overhead issues had not been addressed in the existing multi-agent IDS architectures, but there are only a few studies that deal with this issue.
The proposed architecture in [106] uses a mechanism to keep the number of messages constant for each time interval. Keeping the number of messages constant will delay the communications among all the architecture agents and reduce the system throughput, which is not suitable with a real-time IDS.
Another solution proposed by [67], was to keep the message size small, only 20 bytes. In this technique, the system needs to break the data of the network traffic into smaller packets and send them in multiple rounds. This will cause additional overhead due to a large number of messages, especially when DDoS and Worm attacks launched in the network.
A mechanism used by the architecture proposed in [97], divides the network into segments to allow anti-worm mobile agents spread to clean the infected machines.If there is no worm detected in certain segments for a certain time, the antiworm mobile agents will stop spreading in those segments. This mechanism reduces the overhead system cost in certain cases, but will fail when worm invasions acyually happen, the anti-worm mobile agents spread to clean the infected machines, which will make the matter worst in the infected segment due to the communication overhead among the architecture's agents.
The hierarchical architectures suggested in [19] consists of multiple layers that cause long communication cycles among the architecture agents. The more layers the hierarchy has, the more communication overhead will happen. From this discussion, there are approximately no mechanisms to control the vast communications in the existing multi-agent IDS architectures. The few studies discussed here provide limited solutions to reduce the message size and number during agent communications. We argue that these solutions, also, cannot scale up to vast communications, especially when massive attacks, such as DDoS, strike. Therefore, it is necessary to develop protocols to control the communications in multi-agent IDS architectures to improve the performance of attack detection. Also, it is critical to update the measurement methods for evaluating communications in IDS.

D. RESEARCH QUESTION RQ 4 (WHAT METRICS ARE USED TO MEASURE AND EVALUATE MULTI-AGENT IDS ARCHITECTURES?)
Based on [98], the metrics and measurements used for evaluating the characteristics of real-time distributed IDS systems can be divided into three categories: logistical, architectural, and performance metrics. The logistical metrics are used for measuring characteristics such as manageability, configurability, maintainability, and other platform requirements. The architectural metrics are used in measuring characteristics such as scalability, load-balancing, system throughput, learning, adjustable sensitivity, robustness, and supportability of multi-sensor and multi-analyzer. The performance metrics are used for measuring how well the system is performing, such that analysis capacities, false and accurate detection, recall, accuracy, response time, and traffic latency. Some of these metrics are quantitative, and some are descriptive. The results show that the logistical characteristics of multi-agent IDS architectures were not sufficiently evaluated in the literature, except in [5], [37]. In these architectures, the proposed systems reported they have good anti-destroy, self-restore, and self-configuration abilities.
As previously discussed, there are interrelationships among the characteristics of multi-agent IDS architectures. Therefore, in addition to the performance metrics derived from the confusion matrix such as FPR(False Positive Rate), FNR(False Negative Rate), accuracy, and detection rate. In most of the studies, the architectural and performance characteristics were correlated and evaluated using combined metrics, for measuring the effects of multi-agent features such as coordinated team learning, adaptation, and scalability, on intrusions and attack detection. For instance, the metrics such as network latency, bandwidth consumption, number of data packets per second, and detection rates concerning the number of instances and data packets were used for evaluating the performance of the scalable multi-agent IDS architecture proposed in [6].
To evaluate learning, scalability, and adaptation of the distributed hierarchical architecture proposed for detecting SQL injection attacks in [9], processing time, response time, similarity measure, detection rate, FPR, and FNR were used. The same metrics were also utilized to evaluate the same characteristics in the proposed architectures in [11], [73], [74].
Additionally, a metric named error related to the number of cases that were also used for assessing the fault tolerance of these proposed architectures. The same authors used a similar version of these studies in [72], but for evaluating the system on DoS threats in web services.
For measuring the performance of the adaptive immune multi-agent IDSs in [42], [89], [90], [99], the proposed architectures were evaluated by using comparisons of a value called Mature Context Antigen Value( MCAV), used for measuring danger value, concerning the number of hosts those used to calculate it, bandwidth saturation, network connections, memory loading, and CPU usage. Also, the selfadaptive immune multi-agent IDS architectures in [7], [64] were evaluated by measuring the number of the generated memory cells(attacks data) in ten rounds and the detection rates of all hosts. The bandwidth allocation over time, attack's spread rate and network status during known and unknown attacks, and convergence were used for evaluating the multiagent-based architectures inspired by the human immune system for detecting client's misbehaviour [56], [68]. The TPR and FPR were used to evaluate the evolutionary multiagent approach to anomaly detection and cyber defence [54].
The performance of the adaptive intelligent qualia-based IDS in [38], evaluated using detection accuracy, false detection, precision, and recall. Another measurement of adaptation impacts, on the performance of multi-agent architecture for DoS, was the effectiveness related to the numbers of patterns [43].
Collaborative multi-agent IDS architectures were evaluated using different metrics such as precision that used with the collaborative distributed multi-agent IDS in SCADA (Supervisory Control and Data Acquisition) [10]. Also, the collaborative multi-agent IDS architecture for detecting DDoS, [106], assessed using detection rate, FPR and FNR rates concerning varied numbers of agents and gateways, and the number of times collaborative agents need to communicate with each other in different sized networks. The proposed multi-agent-based architecture in [46] that used coordination and interaction between agents for network audit and attack detection used two metrics, detection time and entropy values of different properties of UDP flooding attacks were used for evaluating.
The cooperative multi-agent architecture for detecting worms [8], is evaluated by using the relationship between the probability of worms to discover new vulnerable nodes, and the percentage of worms payloads with respect to the total payloads sent. Another metric that also used was the percentages of the infected nodes. In [49], [100], the attack traffic, network traffic before and after a filtering process, and botnet propagation were used for evaluating the cooperative multi-agent-based systems against botnets. Also, agent learning rates, botnet presence degree in computer systems, and training errors were used to evaluate the cooperative multiagent system of botnets in [50].
In [16]- [18], metrics such as attack intensity per packet size, energy consumption over time, attacks detection rates to the percentage of attacks, successful detection with respect to the percentage of malicious nodes, number of alive nodes over time, consumed energy over time, the total energy consumption of nodes were used to evaluate the learning of cooperative intelligent agents in detecting and preventing intrusions. Detection rate with respect to the numbers of agents and gateways was also used to evaluate the collaborative architecture in [106].
The adaptive and cooperative multi-agent architecture for botnet detection in [95] used metrics called relationship, response, and synchronization. The relationship metric characterizes the distribution of clients. A too high value of this metric is considered as abnormal. The response metric is the difference between broadcasting requests and receiving responses. The synchronization metric characterizes the synchronicity in the behaviour of clients. The multi-agent distributed information security system that characterized by collaboration and adaptation was evaluated by using metrics such as threat levels of attacks [91]. False responses and non-response rates were used for evaluating the learning, collaboration, and adaptation capacities of the distributed multi-agent intrusion detection architecture by [48].
The study in [65], used suitability value (a value produced by fuzzy logic), botnet presence degree, detection rate, and FPR to evaluate the intelligent multi-agent based approach for botnet detection by using fuzzy logic.
In the proposed architecture in [67], system latency, accuracy with respect to the percent samples tested were used for comparing a bunch of machine learning algorithms integrated with a distributed multi-agent IDS to defend multiagent malicious behaviours. Also, the multi-agent-based architectures for unusual network behaviour detection that integrated with several anomaly detection techniques were evaluated using detection rate, clusters' number, accuracy, FPR [4], [70].
To evaluate the robust and fault-tolerant distributed intrusions detection system by [44], RAM, and CPU usage with regard to users' numbers, detection rate, FPR, and FNR were used. Also, false and negative detection were used For evaluating the proposed architecture in [45], [62].
The multi-agent system for attack classification based on a reputation algorithm was evaluated by using classification accuracy using and without using reputation [60], [76].
As discussed, multi-agent IDS architectures evaluated using logistical, architectural, and performance metrics. The logistical metrics were very rare, while most of the proposed architectures evaluated using combinations of architectural and performance metrics. From this discussion, we can notice the absence of measurements related specifically to multi-agent, such as the metrics used for evaluating team learning. Therefore, it is necessary to implement these measurements and metrics to evaluate multi-agent IDS architecture. Also, proposing new methods and guidelines for using the current metrics to evaluate these architectures.

V. VALIDITY THREATS
The selected studies investigated in this SLR were retrieved using keywords and terms related to multi-agent IDS architectures. Then the retrieved studies were filtered out manually using selection criteria. There may be some risks the selected studies do not reflect the actual state of the art of multi-agent IDS architectures. First, during the retrieval and selection of the studies, some papers possibly were missed out due to the incompatibility of the keywords and terms used in some publications. Second, after retrieving the studies, there may be new publications in the online databases that were supposed to be included for answering the research questions, but they were not. Third, the citations of the listed studies may vary from the actual status of the materials due to the changes in citation numbers everyday. However, recurred search and checking were repeatedly conducted after the retrieval of the studies to see whether there are new studies published or any citations status change.

VI. CONCLUSION AND FUTURE WORK
The purpose of this research was to investigate the existing multi-agent IDS architectures to identify the most challenging limitations that impact intrusions detection performance. In order to achieve that, this research used the protocol described by [34] to conduct SLR in software engineering. The specified protocol was utilized to design the research plan of this study, including the formulation of the research questions. The plan was executed to retrieve, assess, and filter out studies to select the most relevant ones for answering the research questions. The first and second research questions were to identify and categorize the components of the existing multi-agent IDS architectures, and the characteristics that affect the performance of intrusions and attacks detection. Then, the third research question highlighted the issues of the multi-agent IDS architectures. Finally, the fourth research question was to find out the metrics used for evaluating the current architecture to see if lacks of measurements exist. The review was carried out, and the data of the primary studies were selected, assessed, and synthesized. Then, the results were discussed in the context of the formulated research questions. The objectives of this study have been achieved by answering the research questions and identifying the issues in the existing multi-agent IDS architectures.
To conclude this SLR, as discussed, the results of this SLR emphasize that multi-agent IDS architectures have several advantageous characteristics that can help to develop performant IDSs. It is also discovered that there are several issues in multi-agent IDS, exhibited by the selected studies, that can degrade the performance of intrusions and attacks detection. The techniques, mechanisms, and schemes used to deal with multi-agent IDS scalability, adaptation and learning, load balancing, fault-tolerance, and self-management suffer issues discussed previously in this article. For example, most of the multi-agent architectures use supervised learning based on individual agent which is completely impractical to cope with the very rapid growth of network intrusions and attacks.  This SLR also found out that there are limitations in the measurement and metrics used for evaluating the multi-agent IDS architectures.
For future work, we are striving to develop a cooperative learning model for multi-agent IDS architectures. The proposed model is based on a reinforcement learning algorithm to let the agents learn by experience without prior knowledge. The cooperative learning model is to enable the agents to cooperate and learn faster. Additionally, we aim to make an adaptation model to enable the agents to choose the most suitable locations on the network for efficient execution. A new architecture is developed, and the experimental results will be presented. We recommend researchers to adapt suitable agent-based system methodologies to design and develop multi-agent IDSs. Researchers can also use the available network simulation software such as NS-2 and OMNeT++ for testing their proposed models. For example, one can study how a cooperative and adaptive multi-agent IDS using a machine learning algorithm can perform better than the tra-ditional monolithic system. Developers can adapt the tested methodologies and frameworks of multi-agent IDSs to their solutions. In UML and AUML there are several diagrams and tools available for designing multi-agent systems. For implementation, Java Agent-Based Modelling (JABM) and Mesa framework in Python 3+ can be used to develop multi-agent IDSs.

APPENDIX A MULTI-AGENT IDS ARCHITECTURAL PROPERTIES AND CHARACTERISTICS
See Table 3.

APPENDIX B AGENT TYPES EXHIBITED BY MULTI-AGENT IDS ARCHITECTURES
See Table 4. Table 5.

APPENDIX D THE SELECTED STUDIES WITH THEIR RESPECTIVE BIBLIOGRAPHIC INFORMATION
See Table 6. Table 7.

APPENDIX F SOURCES OF THE SELECTED STUDIES
See Table 8. 25