Evaluating the Influence of Twitter Bots via Agent-Based Social Simulation

Social Media is used by many as a source of information for current world events, followed by publicly sharing their sentiment about these events. However, when the shared information is not trustworthy and receives a large number of interactions, it alters the public’s perception of authentic and false information, particularly when the origin of these stories comes from malicious sources. Over the past decade, there has been an influx of users on the Twitter social network, many of them automated bot accounts with the objective of participating in misinformation campaigns that heavily influence user susceptibility to fake information. This can affect public opinion on real-life matters, as previously seen in the 2020 presidential elections and the current COVID-19 epidemic, both plagued with misinformation. In this paper, we propose an agent-based social simulation environment that utilizes the social network Twitter, with the objective of evaluating how the beliefs of agents representing regular Twitter users can be influenced by malicious users scattered throughout Twitter with the sole purpose of spreading misinformation. We applied two scenarios to compare how these regular agents behave in the Twitter network, with and without malicious agents, to study how much influence malicious agents have on the general susceptibility of the regular users. To achieve this, we implemented a belief value system to measure how impressionable an agent is when encountering misinformation and how its behavior gets affected. The results indicated similar outcomes in the two scenarios as the affected belief value changed for these regular agents, exhibiting belief in the misinformation. Although the change in belief value occurred slowly, it had a profound effect when the malicious agents were present, as many more regular agents started believing in misinformation.


I. INTRODUCTION
The influence of social media in our daily lives has increased significantly over the last decade. What began as a social network to share thoughts with individuals has evolved into something bigger, with more active users than ever, including politicians, multi-million dollar companies, and advertising agencies. According to Dean [1], 40% of internet users utilize social media for work. In its current landscape, the impact of what is posted on these social media networking platforms has proven to have real political and economic consequences The associate editor coordinating the review of this manuscript and approving it for publication was Barbara Guidi . that cannot be overlooked. Twitter is one of the largest and most popular social media networks, with over 186 million daily active users, generating over $3 billion in revenue per year [2]. It allows individuals to share their thoughts through tweets using the 280-character text editor, alongside images, polls, videos, and links. It implements a follower-following system, where individuals follow other accounts to read a compilation of their tweets in a streamlined format.
In 2016, Twitter launched the Twitter API, providing developers with tools to implement many of the regular Twitter functions in their code. This allows the creation of many third-party apps and Twitter bots. Twitter bots can post content, respond to others, retweet content, create relationships with other users, and direct message to other accounts. They are autonomous accounts that can function with no human interference and interact in the social media environment alongside human users. While many Twitter bots have beneficial objectives, such as automatically informing their followers about earthquakes, 1 there is still a significant problem with bots designed with malicious intent. Many artificially inflate a tweet's user interaction or an account's follower count, while others are part of misinformation campaigns to spread false information on the social media networking site.
In 2011, many extensively used social media to locate victims of the earthquake that hit the east coast of Japan and reached 9.0 on the Richter scale. A study conducted by Takayasu et al. [3] reported that malicious users took advantage of the event to spread misinformation during the state of the emergency. Another study by Cresci et al. [4] reported that Twitter bots were disrupting the stock market by using the piggybacking technique, where bots are used to increase the stock price of a large company, increasing the price of subsidiaries as well. Also, Takacs and McCulloh in [5] studied the influence of social media in the 2018 U.S. Senate Election and reported that many Twitter accounts engaging with political tweets were bot users, inflating a politicians follower count and the number of user interactions to shape public opinion of candidates (e.g. more followers leading to think they were more popular). Also, a recent study by Allyn [6] at Carnegie Mellon University found that 45% of the accounts discussing the COVID-19 pandemic indicated signs of bot behavior. It is important to notice that even when the misinformation originates from a bot, the spread is mainly done by human users. A study by Zhao et al. [7] elucidated how the propagation of misinformation tends to reach more communities in comparison to real ones. This begs the question: If human users cause most of the propagation, how much influence do bots have on the beliefs of people?
In this paper, we propose an Agent-Based Social Simulation (ABSS) using agent-based systems to develop a social network similar to the Twitter network to study and analyze the influence of malicious bots in spreading fake information. In this regard, we developed three types of agents: Deceptive agents, representing bots or malicious human users; neutral agents, representing most Twitter users; and news agents, official sources who share accurate, verified information. Through this simulation, we investigated the differences between two distinct scenarios, one with and one without deceptive agents. The main objective behind this approach was to understand the impacts that bots have on the neutral agents when encountering false information. This was accomplished by implementing a belief value attribute, which allows studying and comparing how susceptible neutral agents are towards misinformation. To provide an abstract representation of misinformation, we depict the validity of information through three types of tweets that an agent can generate: neutral, real or fake. 1 https://twitter.com/earthquakesLA During our research, we realized that the published articles in the literature discussing this subject of research tend to focus on the spread of misinformation through agent simulations; these attempts fail to address certain characteristics of social media, such as communities (large groups of accounts following similar topics) and different types of agents with distinct behaviors. The significance of our research paper has been in the development of a model that closely mirrors the behavior seen in social media, alongside the belief value. The belief value provides a quantitative way to simulate human beliefs. Our main contributions in this paper are as follows: • The representation of characteristics of social media networks, such as communities, variable number of followers and tweets per agent.
• The validation of the proposed simulation by comparing real-life user interactions with those generated by the agents in the simulation.
• The creation of an approach for comparison between two scenarios to explore how neutral agents interact in a social network with and without deceptive agents.
The rest of this paper is organized as follows: Section II discusses previous related work. Section III provides background information regarding Twitter, bots and Multi-Agent Systems. Section IV gives an overview of the proposed methodology. In Section V we provide the results of the simulation and discuss them in Section VI. In Section VII we validate the data generated by the simulation and, finally, Section VIII concludes and outlines future work.

II. RELATED WORK
Several rumor diffusion models exist based on the Susceptible-Infected-Recovered (SIR) model designed by Daley and Kendall [8] where they compare the rumor spreading to epidemic models, where each agent can be in one of three states: Susceptible, Infected or Recovered. While in the Susceptible state a node is open to be infected by those nodes in the Infected state and, after some time, a cure would spread as well, provided by the Recovered state. While it is tempting to relate the SIR model to rumor spreading in social media, several changes have to be made to accommodate different means of communication and spread of information.
Many authors have expanded on the idea of the SIR model to apply it to Social Media, where misinformation spreading is a large unregulated problem. Serrano and Iglesias proposed a model to validate viral marketing strategies through an ABSS. They explored rumor propagation techniques and created a model to simulate it, called BigTweet [9], which they then released as an open-source software. The main limitation encountered was the ability to handle more than one type of agent, as this is critical for our research focused on malicious users.
Research by Ikeda et al. also use as a base the SIR model to generate an Agent-Based Information Diffusion Model to evaluate data from the previously mentioned 2011 East Japan earthquake. To make the model more robust, they introduced the idea of diversity and multiplexing of information paths [10]. Similarly, research conducted by Okada et al. also provide a more robust version of the SIR model, with data from the same event [11]. The results showed by the latter were compared with real Twitter data to evaluate the susceptible, infected and recovered users using metrics such as number of retweets. Similar to other related works, the rumor propagation is the focus, rather than the origin of that false information, which is something we address in our research.
Kundu et al. approach the information propagation model in their work by developing a novel fuzzy relative willingness model. The diffusion model was able to successfully utilize the external influence factor, as well as the susceptibility of individual nodes to quantify human willingness [12]. While the objective of the paper differs from our research, the implementation of the external influence factor contributed to our research as we applied a similar function as well.
Ross et al. validate the concept of the spiral of silence in their research paper [13] through an agent-based model. The spiral of silence explains how the influence of surrounding negative opinions can affect the spread of positive opinions in a social media environment. The way the simulation was built differs from ours, but it provided insight into how users tend to react while consuming negative media. Wang et al. study information entropy, which incorporates new types of variables to the simulation, including the degrees of trusts agents can set between each other [14]. This work, similar to others, centers on rumor spreading models on social media, but focuses on the interaction between two given agents by adding weights between each node to represent trust.
Research by Yan et al. dives deep into the concept of how retweets and quote-tweets influence the behavior of users in social networks by using the concepts of game theory and developing a reward mechanism [15]. The objective of the research is to study agent behavior given goals, such as a higher number of retweets. This work does not center on rumor spreading or malicious agents, but it proved to be insightful in providing more information about the retweet cycle, which is the means by which information spreads on social media sites, such as Twitter.
Research highlighting bot behavior includes work by Carley [16] outlining the BEND framework as a way of identifying misinformation maneuvers in social media environments. Other studies related to bot behavior can be reviewed in Cresci et al. paper [17] where they compare human behavior in social media similar to DNA sequences and identify bot behavior based on a predictable set of actions. They compare the actions of several Twitter human users and bot users, which they call their digital DNA, and try to identify similarities between them.
Later work by Beskow and Carley highlights two bot misinformation maneuvers: backing and bridging. The former focused on bots interacting with agents with high influence (larger number of edges) to spread false information, while the latter focuses on bridging two communities, with the bots at the center of it. For the model, the authors developed a twitter_sim framework as a way of simulating the Twitter social network. They implemented a belief value assigned to each agent in a way of representing the belief of a real user when encountering misinformation on its environment, along with a similarity weight between agents that has an impact on each agent's beliefs. This belief value defines how dedicated each agent is in believing misinformation based on the type of tweet presented [18].
By looking at the literature, we have become aware of the lack of research underlining the intentional spread of false information by malicious users. While many rumor-spreading models were proposed, most of the work did not provide information on the origin of those rumors nor insight into the effects this misinformation can have on the platform users. We address these issues in our ABSS model by defining different types of agents with different roles, behaviors, and objectives.

A. TWITTER SOCIAL NETWORK
The Twitter network site works similar to other social media platforms, where the contents users consume are derived from the accounts they follow. The primary posting system is called a tweet, described as ''any message posted to Twitter which may contain photos, videos, links, and text.'' 2 Twitter generates a Timeline showing tweets from all followed accounts and offers the option to engage with a tweet by selecting a Like button, commenting on it, sharing it through a retweet to forward the tweet to other users, or using the Quote retweet feature to add additional text while sharing. Twitter calls these user interactions engagements, which are used to measure the total number of users interacting with an individual tweet.
One of the offered Twitter's features is hashtags, symbolized by the character ''#''. 3 This categorizes users' tweets to be part of a much larger conversation by writing a specific tag attached to their tweets. For example: using the ''#Covid19'' hashtag can direct users to tweets related to the Coronavirus pandemic. Similarly, tweets can also contain cashtags (using the character ''$'') as a way of categorizing conversations related to the stock market, such as ''$AMZN'' when referring to Amazon. Another Twitter feature is the Verified mark used to differentiate those users whose identities have been verified officially by the company. This mark commonly applies to to public figures, such as celebrities and politicians, to avoid impersonators on their platforms.
Twitter implements an algorithm to display contents relevant to its users, called Top tweets. While the inner features of this algorithm are not open source, Twitter claims that the targeted tweets are chosen by the tweet's popularity (high engagement) and keywords relevant to the users. 4

B. TWITTER BOTS
As stated by Storey and Zagalsky [19], a software bot's objective is to automate actions by either engaging in repetitive tasks, providing assistance through a chat function, or simply simulating a human user. Lebeuf [20] categorizes bots based on their intelligence level and purpose. When it comes to intelligence, a bot can be characterized based on its adaptability to the environment, reasoning to rules, and autonomy from human users. Based on their purpose, they can be categorized as Generalist bots that have a wide range of abilities; Transnational bots that work for users; Informational bots that gather information for users; Productive bots that execute time-consuming tasks on behalf of users; Collaborative bots that improve users communication.
Roth and Pickles [21] describe Twitter bots as automated accounts with the same privileges as regular Twitter users, such as creating, sharing, or engaging with tweets. Their use in the Twitter social network is not inherently malicious or prohibited, and it is even encouraged by the company for customer service applications.
Twitter bots are developed through the Twitter API, allowing developers to automate the creation of tweets, respond to comments, and engage with other tweets through likes and retweets. What sets bots apart are based on the behaviors developers implement in them. As mentioned by Roth and Pickles, the real concern is platform manipulation, which is described as an alteration of the public conversation happening on the social network through fake engagements due to their impact on the human user's perception of public opinion.
Jamison et al. [22] study the common behaviors of Twitter bots and categorize them based on objectives. Some common characteristics of a malicious Twitter bot include extreme marketing campaigns, an inflated number of fake followers, a large number of scheduled tweets, or high levels of retweeted content. However, malicious Twitter users can also be human and present behaviors such as engaging heavily in political content, negative/opposing views, or spreading (retweeting) similar content from other sources.

C. MULTI-AGENT SYSTEMS
The simulation presented in this paper, at the base level, is developed as a Multi-Agent System (MAS) [23]. As described by Kinny and Georgeff [24], MAS environments consist of a series of agents: reactive, autonomous, and internally motivated entities with a defined behavior that can be altered during its execution based on internal or external factors. The simplest example of an agent-based system can be seen in a thermostat, activating heating or cooling systems based on what temperature it captures from the environment.
The interaction between agents is what sets MAS apart from other object-oriented methodologies. These agents can interact with one another, with the environment or directly with a human user, which can lead to changes in its behavior. Figure 1, demonstrates a high-level structure of multi-agent systems. In MAS environments, agents require a series of characteristics that define their behavior in an environment. As explained by Slhoub et al. [25], these are: Autonomy (able to act on their own), Reactivity (acts based on what it perceives from the environment), Proactivity (works to achieve a specific goal) and Sociability (interacts with other agents) By definition, many common factors exist among agents and bots, but what sets bots apart is that they are typically implemented in environments where humans interact, as mentioned by Lebeuf et al. [20]. For this reason, nonhuman users are typically described as ''bots'' when it comes to social media, while the ''agent'' term is used more predominately in simulation-based experiments.

D. GRAPH THEORY
A graph, in its simplest form, is a set of points (vertices) connected by a set of lines (edges), as defined by Gibbons [26]. Any graph G, regardless of what they represent, is given by the formula G = (V , E), with V representing the number of vertices and E the number of edges.
Graphs are commonly used to represent social networks due to similarities in their design; vertices represent people, and edges depict the connection between them. For this reason, graphs are widely used when simulating epidemic spreading models and social media environments. Graphs are also used in other fields, such as contact tracing, cybersecurity (interaction between IPs), fraud detection, recommendation algorithms, and network routing. These are connected by the relationships between the nodes and are perfect examples of how information can be presented.
Several types of graphs can be used to display the data based on what is being evaluated. It can be as simple as presenting only nodes and connections, while it can also be weighted (looking for the shortest/longest path). There are also bipartite graphs (for recommendation systems), hypergraphs, trees, and property graphs.

IV. THE SIMULATION DEVELOPMENT
As shown in Section II, related work focused on the spread of gossip and fake information through an environment built only from neutral agents, and how a single tweet propagates through the network. This study will focus on deceptive agents representing malicious users and automated bot accounts created with malicious intentions, and how much their actions affects the beliefs of the neutral agents. To achieve this, we introduce a belief value assigned to each neutral agent as a way of modeling its susceptibility from unofficial information sources. At the end of the timed execution of our simulation study, we inferred the influence based on the difference in the shift in the belief value from the initial parameters set for the environment.
To develop this ABSS, we defined the specifications of the system based on recommended practices defined by Slhoub et al. [25], where we assign each agent a series of properties based on their expected behavior. Next, we collected real data from Twitter to provide validation to our model. With the definition of all agents complete, we developed the model through the use of the Soil Social Simulator [27]. Figure 2 demonstrates a high-level structural view of the proposed model.

A. AGENT DEFINITION
Through the use of an Agent-Oriented Modeling technique introduced by Slhoub et al. in [25], we manage to describe the relationships between the agents to define their behavior. Here, we begin by classifying each agent that will be part of our ABSS and then defining their behavior based on their current state.

1) BELIEF VALUE
One of the fundamental pillars of this simulation is the belief value. Similar to its use by Beskow and Carley [18], we implement this property to determine the susceptibility of neutral agents to fake tweets, ranging from 0 (believes only in official tweets) to 1 (susceptible to fake tweets). Alongside its susceptibility, we also utilize the belief value in changing neutral agent states. When a threshold is met, the behavior of the neutral agent changes. For example, a belief value over 0.7 does not only indicate the user is more prone to sharing misinformation, but also it has a higher chance of generating fake tweets as well.
The belief value shifts after a neutral agent engages with a tweet. An official tweet causes the belief value to be reduced, while a fake tweet increases it. Neutral tweets can also be generated, causing no impact on the neutral agent's belief value. Whenever a neutral agent engages with a tweet, its effect on belief value is applied, regardless of the fact, whether it is neutral, fake or official. Through the belief value attribute, we simulate a human's beliefs.

2) AGENT CLASSIFICATION
All agents are modeled after a parent class, with each subclass representing a type of agent. This is due to the three types having access to the same functions, including sending, engaging, and sharing tweets.

a: NEUTRAL AGENTS
This agent represents the average Twitter user. It interacts in the environment through tweets created or shared by other agents. Each neutral agent has a belief value property that determines how susceptible it is to the tweets posted in the environment. This may change depending on the agent's perception of the content; the higher the belief value is, the higher chance of engaging and creating fake tweets. The characteristics of this agent are Autonomy, Reactiveness, and Sociability.

b: DECEPTIVE AGENTS
Represents malicious users in the Twitter network. They aim to post fake tweets and distribute them to the utmost extent in the environment. Neutral agents do not differentiate between neutral and deceptive agents. The characteristics of this agent are Autonomy, Proactiveness, and Sociability.

c: NEWS AGENTS
These agents represent news outlets on social media. These only generate official tweets, which represent verified news stories. Additionally, these agents do not engage with other tweets. The characteristics of this agent are Autonomy and Sociability.

3) AGENT BEHAVIOR
Each neutral agent is given a set of states that can shift during the duration of the execution, presenting different behaviors on each time step of the simulation. The state of each agent depends on the belief value or the type of agent. We developed a scale utilizing 5 different behaviors for the neutral agents, ranging between 0 and 1. As the neutral agents interact in the environment, their belief value shifts, and once it reaches a threshold, the neutral agent changes states. Figure 3 illustrates the different states that can be reached by neutral agents based on their current belief value.

a: NEUTRAL STATE
A neutral agent is in the Neutral State when its belief value is between 0.375 and 0.625. While in this state, the agent can only send neutral tweets.

b: FAKE BELIEVER STATE
A neutral agent is in the Fake Believer State when its belief value is between 0.625 and 0.875. While in this state, the agent has an increased probability of sending fake tweets itself.

c: INFECTED STATE
A neutral agent is in the Infected State when its belief value is between 0.875 and 1. While in this state, the agent can only share and post fake tweets.

d: TRUTH BELIEVER STATE
A neutral agent is in the Truth Believer State when its belief value is between 0.125 and 0.375. While in this state, the agent has an increased probability of sending and sharing official tweets.

e: VACCINATED STATE
A neutral agent is in the Vaccinated State when its belief value is between 0 and 0.125. While in this state, the agent has an increased probability of sending and sharing official tweets.

f: DECEPTION STATE
This state is exclusive to deceptive agents. During this state, the agent can only post fake tweets.

g: NEWS STATE
This state is exclusive to news agents. While in this state, the agent can only send official tweets, with a high probability of posting.

B. DATA COLLECTION
The Twitter data collection was carried out in four steps, as referenced in Figure 2: first, we extracted a tweets dataset through Twitter's API V2 containing the word ''vaccine'' from December 2021. We then analyze the data to obtain metrics regarding tweets, engagements and follower count. Next, we take a sample from the accounts listed in the dataset and processed them through the Botometer API [28] and, finally, obtain metrics regarding bot behavior in the accounts analyzed.

1) TWEET EXTRACTION
Utilizing the Twitter API V2, 5 we compiled a dataset with over 200,000 tweets containing the word ''vaccine'' from the 9th to 18th of December 2021. Each tweet was extracted 3 days after its initial posting date to collect the number of retweets, likes, quotes-tweets and generated comments. Additionally, we collected public user information for each user, including the number of followers, following accounts, verification status and tweets. A description of the structure of the collected data is shown in Table 1  Our data collection process was created due to unavailability of datasets that provide the esential parameters needed for the proposed methodology, such as engagement metrics. The dataset was collected over several days with publicly available information regarding the accounts extracted and numbers related to engagements. In addition, since part of our analysis focused on collecting engagement metrics, we could not extract tweets as soon as they were posted, as a result, each collected tweet had to be at least 3 days old.
We were able to collect 220,085 tweets from 150,692 accounts. Tweets were filtered only by the keyword ''Vaccine'', excluded retweets and, as a result, were not part of the same thread. Our motivation was to obtain a set of metrics from tweets that we could use to compare with the results given in the simulation. In essence, this comparison would validate our simulation. We aligned our simulation to provide similar results to the data we collected, which can be seen in Section VII.

2) DATASET ANALYSIS
We calculated the correlation coefficient (r) between the number of followers and the number of engagements each tweet received. We noticed a weak correlation between both, with r = 0.03913. As a result, we did not consider this a deterministic factor in our simulation. When considering the number of tweets per agent, the dataset gave a moderate correlation between the number of followers and the number of tweets, with r = 0.374347.
During our analysis we found that the average number of followers for verified and non-verified accounts were highly different, with 336,191.3748 and 15,371.5414 followers, respectively. Although, some outliers were found, such as, follower counts going to as low as 14, the data showed that a high number of followers associated with verified accounts. We used these findings as the basis for the news agents in our simulation.

3) BOTOMETER
We took a sample of 10,420 accounts from the initial tweet dataset to be analyzed through the Botometer API [28]. This API analyzes the public information of each provided account and gives a score based on its activity on a scale of 0 to 5. The scores generated by the Botometer API are: Echo-chamber, for accounts that participate in follow-back groups (used by accounts wanting to increase their follower number by following other users back) and engage heavily with political tweets. Fake Follower, for accounts with a high number of bot followers. Financial, for bots using cashtags. Self-declared, for accounts that appear in botwiki.org. Spammer, for accounts labeled as bots in large Twitter data sets. Other, which takes into account miscellaneous manual reports or user feedback from Botometer users.

4) BOT ANALYSIS
Out of the 10,420 accounts analyzed, 164 were private or deleted accounts that could not be accessed, and 1,637 displayed an overall score of 3 or more. The highest scores were observed from the Other category, with 993 accounts over a score of 3, followed by Echo Chamber with 627 and Fake Followers over 299. While the authors of Botometer do not recommend binary classifications due to the nature of how scores are generated, we decided to focus on those with a score higher than 3 as a starting point for the number of deceptive agents in our simulation.

C. SIMULATION ENVIRONMENT 1) FRAMEWORK SELECTION
Based on the agent requirements established in Section IV-A, we decided to research existing frameworks to simulate social network environments that allow us to configure the agents in the simulation and their behavior. The frameworks BigTweet by Serrano et al. [29] and Soil by Sánches et al. [27] were tested, but the latter was chosen due to its customization options.
Soil is an ABSS developed in Python by Jesús Sánches, Carlos Iglesias and Fernando Sánches-Rada at Universidad Politécnica de Madrid, focused on simulating social network environments. The Soil Framework has three main elements: The network topology, the agents, and the environment. Each agent will have an assigned behavior based on its current state. In Soil, each agent is given a default state, and then based on the actions it can perform while on that behavior, it can change states. As an example, while simulating a SIR environment, agents can face a probability of infection while on the susceptible state; if this happens, the behavior of the agent changes, as it can now infect neighboring agents.
Soil uses JSON or YAML files for configuring the simulation. In this configuration, developers can set a series of values that will affect the execution of the simulation, such as the number of intervals, types of agents present in the simulation, and global environment parameters (e.g. the probability of spreading). By executing the configuration file, Python begins simulating the environment depending on the number of time-steps decided by the user. After it completes the simulation, the output is presented through a CSV file and a dynamic graph that can be displayed using a graph analysis application, such as Gephi. 6 The simulation initialization depends on the configuration provided in the YAML file, which denotes what types of agents are part of the simulation, their weight, the total number of agents, and hyper-parameters that affect the execution. The multi-agent system is given by the graph G = (N , E), where N represents the user agents and E represents the indirect edges representing the following/follower of each node.
When the simulation is executed, Soil generates agents based on the weights assigned to each type in the configuration file. Following their creation, the agents are assigned one of seven topics, with the objective of creating communities (or clusters). When selecting each neutral agent's following accounts, we base it mainly on proximity to the chosen topic and making sure each agent follows between 1% and 5% of the total number of nodes to maintain an average path length between 2.5 and 3.0. This allows information to travel fast within a community, where users are more tightly coupled, but slower between agents on different topics. While focusing on maintaining clusters, agents can follow users outside their topics to allow communication between them.
Each neutral and deceptive agent was assigned a probability of tweeting, which correlates with the number of followers. The probability of tweeting is calculated by (1), where P t represents the probability of tweeting, i shows the number of followers of a specific agent, and j presents the number of followers of every agent: Finally, when starting the simulation, each neutral agent is assigned a belief value between 0.375 and 0.625. This is performed to allow all agents to be initialized on the neutral state, which provides a good comparison point as the belief value shifts throughout the simulation.

3) RUNNING THE SIMULATION
During each time step, each agent can send a tweet based on the probability discussed in the previous section. Depending on the type of agent and its current state, the tweet can be of three types: Neutral, Official, or Fake. Regardless of the agent's type, it is sent to the retweet function, where the spread is calculated and counted. This is defined using two variables: Impressions and Engagements. The former focuses on the number of agents that have processed the tweet either by being a direct neighbor of the account or in contact with it after a neighbor shared it. The latter focuses on the number of agents that engage with it.
We decided to include the retweet function to simulate how quickly information spreads over Twitter. In Section V, we take a closer look at the numbers each tweet generates. This function is based on a recursion procedure, so every time a user shares a tweet, the same function will be called, and it will work under the assumption that every single agent will read all tweets presented in their timeline.
For the retweet function, we calculate two values per agent: the probability of engagement (P e ) and the probability of retweet (P r ). Both require the environmental variables Prob f and Prob r as a way determining the chance of spreading, based on it being fake or real information, respectively, and with x representing the agent.
The engagement probability values are used to determine if an agent believes the tweet presented. This variable can represent a like, comment or retweet, which indicates that the agent agrees with the information presented. The use of the environmental variables Prob f and Prob r is necessary to maintain a controlled number of engagements per tweet, as only considering the belief value will have different results from the dataset collected in section IV-B. The formulas for calculating the probability of engagement are shown in (2) and (3).
Fake news engagement: Real news engagement: After engaging with a tweet, each agent can use the retweet function to forward the same tweet to its followers. This will result in the same function being called, but with the sharing agent as the parameter. The formula for the retweet probability is shown in (4) and (5) Fake news retweet: Real news retweet: A neighboring agent affected by a tweet will result in the agent's BeliefValue increasing or decreasing if the tweet was fake or authentic, respectively. After this, based on the probability of retweeting, agents might share that tweet with their neighbors, repeating the cycle. When the BeliefValue reaches a threshold, the agent changes to one of the states defined in section IV-A3. A graphic representation of the retweet function is demonstrated in Figure 4.
Inside the configuration, we implemented an external influence function. Given the same probabilities explained above, at the start of every time step, each neutral agent's belief value can be affected from outside the Twitter simulated environment, as it does not involve tweets. This allowed us to simulate external influences affecting human users outside the Twitter social network.
The final step when sending a tweet involves the Targeted Tweet function, working as a way of applying the Top Tweets feature used in the Twitter network. If the tweet reaches a large number of engagements in the simulation, it will be deemed a viral tweet. When this happens, the tweet will be shown to all agents in the network that share the same topics with the sender agent, even if an edge does not connect them. Details on how the Twitter algorithm works are unknown to the public, so we work under the assumption that all users use this feature, and the factors taken into account are the number of engaged users and topic similarity.
To summarize, we've included the pseudocode for the decision making algorithm for neutral agents in the neutral state in Algorithm 1 and neutral agents in the fake believer state in Algorithm 2, as an example of how the different states modify the behavior of neutral agents. We designed two scenarios to execute the simulation, with and without deceptive agents. Through this comparison, we were able to study the effect malicious users had over the neutral agents and evaluate the differences between external and internal influences.

A. SCENARIO 1: SIMULATION WITH DECEPTIVE AGENTS
The simulation consisted of 1,000 agents through 250 timesteps, with 70.9% neutral agents, 19.9% deceptive agents, and 9.2% news agents along with an average path length of 2.67. Figure 5 presents a sample of the distribution of agents at the end of the simulation, with each cluster representing the seven topics. We chose this distribution because we wanted to approximate our simulation as much as possible to the data extracted from Twitter. Starting with deceptive agents, we chose a 15% distribution as that was the percentage of accounts with bot behavior from our findings with the Botometer API. For news agents, we started with 5%, as those were approximately the number of verified accounts from the collected dataset. However, as we validate in Section VII, we had to fine-tune these numbers to remove bias and provide a better comparison point with the number of tweets created.
130,255 tweets were sent throughout the 250 time-steps, most of them of were neutral tweets that do not alter the belief value. While the official and fake tweets reached a similar number of agents, the latter had more engagements per tweet. Table 2 and Figure. 6 illustrate the distributions per type of tweet, with the average number of impressions and engagements.  Even with most tweets being neutral, the effects of the fake tweets altered the belief value of the neutral agents, with 9.66% of agents in the Fake Believer state and 4.01% in the True Believer state at the end of the simulation. Figure 7 and Table 3 exhibit the average belief value per time step.

B. SCENARIO 2: SIMULATION WITHOUT DECEPTIVE AGENTS
In scenario 2, deceptive agents were not considered for the execution, keeping only neutral and news agents active in the environment. We implemented the external influence function that simulates changes in belief outside the Twitter network, with the same probability of being affected as a regular fake tweet. Our intention was to simulate an external face-to-face human interaction, as those can affect the beliefs of users as well.
We decided to modify the scenario by removing deceptive agents, keeping 800 agents instead of maintaining the same distribution, and replacing them with neutral agents. The official distribution for this scenario was 88.75% neutral and 11.25% news agents. The average path length was 2.72. An example of this distribution at the end of the simulation can be seen in Figure 8 In the chosen sample, 106,133 tweets were sent, with the majority in number and engagements being neutral. Only 1 fake tweet was sent by a neutral agent during this time,  which reached 31 users, but no engagements. The distribution in scenario 2 is illustrated in Table 4 and Figure 9.
Despite no deceptive agents present in the simulation, 0.32% managed to reach the Fake Believer state and the engagements with official tweets only improved slightly compared to the previous scenario. Figure 10 shows the average belief value per time step, which curved on the opposite direction, alongside Table 5 displays a sample of time steps.

VI. DISCUSSION
We implemented the simulation with two distinct scenarios: scenario (1) utilizes deceptive agents representing malicious users in the Twitter social network and sending false tweets to misguide neutral users. Scenario (2) removes deceptive agents and adds an external influence function to represent external factors that might alter a neutral agent's beliefs outside the Twitter network. By implementing the belief value, we were able to quantify the susceptibility of neutral agents encountering misinformation in the environment.
The majority of tweets generated by the agents during the simulation were neutral tweets, with the highest number of both impressions and engagements, described in    section IV-C3. The number of fake tweets and official tweets generated by agents remains very similar, but for different reasons as illustrated below: • News agents exist in smaller quantities than the other types of agents inside the simulation, but their rate of official tweets were very high.
• Deceptive agents, on the other hand, had a similar rate of sending fake tweets as neutral agents, but there were less deceptive agents than neutral agents.
• Neutral agents could also send official tweets and fake tweets, but at a lower probability. As a result, neutral agents didn't alter the number of fake or authentic tweets in an significant way.
During the execution of scenario (1), fake tweets sent by deceptive agents never reached a viral tweet status (described   in section IV-C3), although several managed to generate over 300 impressions out of the 1000 total agents in the simulation. Deceptive agents focused their efforts on spreading large quantities of fake tweets to affect the belief value of neutral agents, instead of aiming to get a high number of engagements.
Despite the removal of the deceptive agents from the simulation in scenario (2), the results between the two scenarios had a similar number of engagements, impressions, tweets, and the overall standard deviation in belief value, mainly over the early stages. This indicates that the effects deceptive agents had on neutral agents were more evident the more time passed in the simulation.
The average belief value of the neutral agents in scenario (1) increased exponentially during the later stages of the 250 time-step simulation due to the large number of the fake tweets plaguing the simulation by the deceptive agents, with the highest belief value reaching 0.7753. As explained in section IV-A, neutral agents in the Fake Believer state have the option to also send fake tweets, which resulted in 256 fake tweets sent by neutral agents, generating over 33,000 engagements total. This spotlights the significant influence deceptive agents had on the neutral agents, altering their beliefs, as fake tweets sent by neutral agents also affect other neutral agents, increasing the average belief value throughout.and turning them into malicious users.
Throughout the execution of scenario (2), we witnessed a decrease in the average belief value due to the removal of the deceptive agents. However, with the external influence function into effect, an increase in the belief value of the neutral agents was still occurring at a lower rate. The influence of fake tweets on the neutral agents in scenario (1) was more substantial than the external influence due to the higher rate at which neutral agents encountered fake tweets. This enables further comparison between the influence in beliefs between face-to-face human interactions and social network interactions. Information travels faster on social networks, so beliefs have a greater chance of being altered.
One key element observed in scenario (2), despite the removal of the deceptive agents, was the presence of fake tweets. During our tests, the neutral agents ended up in the Fake Believer state, being able to create fake tweets, although in fewer quantities than in scenario (1). We believe that removing malicious users from the Twitter network will reduce the number of fake tweets but not resolve the problem of misinformation being spread.
During the simulation, we worked under the assumption that the neutral agents perceive all tweets in their surrounding environment. Despite this, we observed a slow shift in belief value, as tweets did not display high number of engagements on average. Not a single agent reached the Infected or Vaccinated state (discussed in section IV-A3) during the 250 timestep simulation, which indicates that it takes longer for the neutral agent's beliefs to be altered.

VII. VALIDATION
This section aims to compare the simulation results with those from the dataset presented in Section IV-B to validate our configuration and ensure it is closely related to the data extracted from the Twitter social network. Due to the limitation of being unable to measure a human user's beliefs in a verifiable way, we focused our efforts on validating engagement numbers related to the collected tweets. Our efforts were centered around accurately representing the number of tweets generated during the simulation, alongside an accurate number of engagements to simulate how the neutral agents would react to tweets displayed in their timeline. Throughout our analysis, we found that 40% of tweets did not have a single engagement, regardless of their verified status or the number of followers. To scale it accordingly, we worked under the assumption that a viral tweet is represented by those with over 500 engagements in the Twitter social network and those with over 2.5% of the total number of agents in the simulation. We included this comparison to bring validity to the conducted simulation.
As mentioned in Section IV-C2, we included a global probability variable alongside the belief value to manage the rate at which the neutral agents would engage with tweets. Throughout our tests, utilizing only the belief value would result in a more significant number of engagements compared to the collected dataset. For this reason, we included this variable to correct the engagement flow and maintain it closer to reality. Our validation results can be observed in Table 6, displaying a comparison of the tweets collected with the ones generated by the social simulation with different configurations.

VIII. CONCLUSION
We successfully developed a multi-agent system simulating the Twitter social media network through the Soil Framework. With it, we studied the differences in the social network when bots and malicious users are inserted into the simulation and how the belief of the users are affected by the misinformation shared through it. We simulated two distinct scenarios, one with malicious agents and one without them, to compare how much influence they have on the neutral agents. Also, to evaluate it alongside an external factor in order to find which one was more influential towards the agents. The two scenarios were executed and showed similar outcomes regarding agent interactions with tweets. However, the removal of deceptive agents drastically reduced the average belief value, which leads us to conclude that removing bots from the Twitter platform will not solve the misinformation issue, but would greatly reduce it. The fake tweets' influence over the neutral agents was weak but still effective with the passing of time in the simulation.This is true regardless of the applied scenario, as fake tweets were created by neutral agents in an environment eliminating deceptive agents.
By extracting the Twitter dataset, we ensured the validity of the proposed model by comparing key elements, such as the number of engagements per tweet and bots in the environment. This contributed to the creation of an accurate simulation that we were able to verify against the actual tweets from the Twitter social network.
For future work, validating the belief value in the natural Twitter social network, based on how susceptible users of the social network are when presented with false information, would significantly improve the accuracy of the simulation. In this paper we used the umbrella term engagements to simulate a user's perception being affected by a tweet; future work can approach with more granularity, defining differences in Likes, retweets, Comments and Quote-tweets, as the fourth one is typically used in the social media networking site to correct false information. Additionally, implementing a likelihood function to neutral agents interacting with tweets on similar topics can improve the simulation's accuracy and more realistically measure the neutral agent's probability of engagement.
While reviewing the simulation results in scenario (1), we observed several neutral agents shifting towards lower belief values despite the average belief value trend going upwards. For future work, a deeper psychological study could provide a better understanding of the behavior of the neutral agents maintained firm beliefs on official tweets, despite being in an environment plagued with fake tweets.
ALDO AVERZA was born in Panama City, Panama, in 1995. He received the B.S. degree in systems and computer engineering from the Technological University of Panama, in 2017, and the M.S. degree in software engineering from the Florida Institute of Technology, in 2022. From 2018 to 2020, he worked as a RPA Consultant at Bprosys Inc. He is currently working as an Automation Specialist at Dell Technologies. He was awarded a Fulbright Scholarship by IIE and the United States Embassy in Panama. His research interest includes agent-based systems, with an emphasis on social network behavior. He is a part of the Phi Kappa Phi Honor Student Society.
KHALED SLHOUB received the B.S. degree in computer science from the University of Benghazi, Libya, in 1999, the M.S. degree in computer science from the University of New Brunswick, Canada, in 2008, and the Ph.D. degree in computer science from the Florida Institute of Technology, USA, in 2018. He is currently an Assistant Professor at the Department of Computer Engineering and Sciences, Florida Institute of Technology. His main research interests include software engineering and agent-based systems. His research has centered on developing standard frameworks for formalizing the development processes of agent-based systems. He is currently focusing on studying and analyzing the quality of existing agent-oriented methodologies to provide unified agent-oriented development approaches that can deliver practical means in industrial settings. He is also working on developing a framework to enable understanding and flagging disruptive behavior of distributed social agents by deploying policing agents to determine risks. In addition, his research is focused on finding effective testing approaches to verify autonomous systems. He mainly focuses on finding ways to verify the irregular behavior of agent-based systems.
SIDDHARTHA BHATTACHARYYA (Senior Member, IEEE) is currently an Associate Professor at the Florida Institute of Technology. His Assured Safety Security and Intent with Systematic Tactics (ASSIST) Research Laboratory, primarily conducts research in the area of formal methods for the design, verification, and validation of intelligent autonomous systems, avionics, cyber security, and trust. Previously, he was a Sr. Research Engineer at Rockwell Collins' Advanced Technology Center, where he worked on research programs for assurance of safety critical systems. He was also a Summer Faculty Research Fellow at Oak Ridge National Laboratory and a Summer Research Assistant at the Applied Research Laboratory, Penn State. VOLUME 10, 2022