Implementation of a Multi-Approach Fake News Detector and of a Trust Management Model for News Sources

Technological development combined with the evolution of the Internet has made it possible to reach an increasing number of people over the years and given them the opportunity to access information published on the network. The growth in the number of fake news generated daily, combined with the simplicity with which it is possible to share them, has created such a large phenomenon that it has become immediately uncontrollable. Furthermore, the quality with which malicious content is made is increasingly high so even professional experts, such as journalists, have difficulty recognizing which news is fake and which is real. This paper aims to implement an architecture that provides a service to final users that assures the reliability of news providers and the quality of news based on innovative tools. The proposed models take advantage of several Machine Learning approaches for fake news detection tasks and take into account well-known attacks on trust. Finally, the implemented architecture is tested with a well-known dataset and shows how the proposed models can effectively identify fake news and isolate malicious sources.


Implementation of a Multi-Approach Fake News
Detector and of a Trust Management Model for News Sources Claudio Marche , Member, IEEE, Ilaria Cabiddu , Christian Giovanni Castangia , Luigi Serreli , Graduate Student Member, IEEE, and Michele Nitti , Senior Member, IEEE Abstract-Technological development combined with the evolution of the Internet has made it possible to reach an increasing number of people over the years and given them the opportunity to access information published on the network.The growth in the number of fake news generated daily, combined with the simplicity with which it is possible to share them, has created such a large phenomenon that it has become immediately uncontrollable.Furthermore, the quality with which malicious content is made is increasingly high so even professional experts, such as journalists, have difficulty recognizing which news is fake and which is real.This paper aims to implement an architecture that provides a service to final users that assures the reliability of news providers and the quality of news based on innovative tools.The proposed models take advantage of several Machine Learning approaches for fake news detection tasks and take into account well-known attacks on trust.Finally, the implemented architecture is tested with a well-known dataset and shows how the proposed models can effectively identify fake news and isolate malicious sources.Index Terms-Fake news detection, machine learning, prebunking, trustworthiness management.

I. INTRODUCTION
T HE technological development combined with the evo- lution of the Internet has made it possible to reach an increasing number of people over the years.The spread of smart devices has allowed users to be able to connect anywhere and anytime to the network: the visible advantages are represented by the opportunities for everyone to access information published on the network, easily increase their cultural background, and make their opinion heard.
This scenario has allowed the birth and creation of new websites that provide large amounts of information, even free of charge, to an ever-growing audience eager to expand their knowledge.However, the simplicity with which it is possible to publish news online has allowed anyone to disseminate news of all kinds so that also the propagation of distortions, alternate realities, and lies has increased.This phenomenon is now known as Fake News, so finding reliable information on the Internet has become problematic.
Fake news is defined as information that is partially or completely false, disseminated intentionally or unintentionally through any means of communication that presents an apparent plausibility and a greater increase in the prejudices that lie with it [1].
Detection algorithms have the crucial task of implementing technical approaches in service provisioning and methodologies to aggregate a variety of information in order to infer the reliability of news the user wishes to interact with.However, debunking is a difficult task and has to overcome several challenges: aside from the size of published fake news to be verified, corrective information can sometimes provoke a so-called "backfire effect" in which respondents more strongly endorse a misperception about a controversial political or scientific issue when their beliefs or predispositions is challenged [2]; finally, debunks do not reach as many people as fake news, and they do not spread nearly as quickly [3].
To this, the goal of this paper is not only to evaluate news items, i.e., to understand if the news is real or fake, but also to develop a prebunking system [4], i.e., the process of debunking lies, fake news or sources before they strike, by evaluating the trustworthiness of the news providers.
Trust is tied to the concept of reputation.Indeed, trust can be gained on both direct and indirect bases, but in large networks such as the Internet, it takes time for a user to collect enough direct experience so an entity has to rely on the perception of other entities, that is the reputation.Through reputation, it is possible to collect, distribute and aggregate feedback about participants' past behaviour and then provide a global perception of an entity.This concept enables newsreaders to rely on the community's reputation to identify trustworthy sources, eliminating the need for a trial-and-error approach.To this, in this paper, we have developed a trust and reputation management model so that it is possible for users to understand which are the news providers that can lead to successful collaboration, i.e., that can provide reliable news.This paper is part of the project FAKE, developed as a cascade call of the EU's project TruBlo [5].In particular, this paper provides the following contributions: 1) First, we proposed a detection algorithm that analyses news' written text and classifies the news as fake or real according to several parameters, which include the writing style, fact-checking, sentiment analysis of the text, and the context of the news.2) Second, we develop a trust management model for evaluating news sources, which uses novel parameters, namely expertise, relevance, goodwill, and coherence, to defend against malicious behaviours.3) Finally, we simulate the implemented architecture by using a Kaggle dataset, which contains a total of 20,387 news from various domains (such as politics and economics) to show the performance of each module of the algorithm and its overall accuracy in identifying fake news.The rest of this article is organised as follows: Section II presents a brief survey on fake news detectors and on trustworthiness algorithms used to classify news providers.In Section III, we define the system architecture and the reference scenario.Sections IV and V present the fake news detector and the trust management algorithm.Furthermore, the system performance is analysed in Section VI, while Section VII presents an alternative technology for storing and retrieving information related to the news, namely Blockchain, and compares it to a traditional database.Finally, Section VIII draws final remarks.

A. Fake News Detectors
In recent years, there has been a significant focus in the literature on analysing fake news, and numerous works have been proposed to detect them [6].The growth of social media and the abundance of online information has considerably added complexity to this challenge [7].When sharing news, people often fail to consider the possibility of fake news and tend to believe only the news that confirms their pre-existing beliefs.This lack of critical thinking leads to a failure to reflect on the reliability and truthfulness of the information they see on social media platforms [8].Another issue concerns the rapid spread of fake news, which can propagate much faster, deeper, and broader than accurate news, resulting in a significant proportion of the information people encounter daily being false [9], [10].Furthermore, although fake news is not a new phenomenon, it is rapidly increasing and gaining public attention [11]; the leading cause is that fake news can be created cheaper and faster than traditional news media [12].In this regard, fake news detection is becoming a critical mechanism that proposes to detect fake content as fast as possible and provide assistance to journalists and fact-checkers [13].Below, we want to analyse and classify the most important detectors based on their techniques and approaches.
In these terms, two well-known fake news detectors, based on analysing the news features through multiple machine-learning techniques, are illustrated in [14] and [15].In the first work, the authors make use of different machine-learning approaches as classifiers for fake news considering linguistic and count-based features, such as length and word count.Authors denote how fake news articles usually tend to be shorter, appear more negative, and adopt a more personal disclosing tenor.In the second work, the authors propose a similar solution and demonstrate how leveraging various sources of sentiment, e.g., images and visual media, can be used to improve accuracy.The approach is evaluated using several datasets and similarity techniques.Both works obtain the best results with the Support Vector Machine (SVM) algorithm, which is then used for the fake news detection processes.Moreover, an approach mainly based on SVM is presented in [16], in which the authors propose a fake news detection model based on n-gram analysis, i.e., an approach used in language modelling and natural language processing, combined with a linear SVM (LSVM) classifier.Various sequences of characters or words, namely n-grams, are generated from a training set and compared in order to classify fake from honest news.All the n-grams are then used as input for a machine-learning technique responsible for the final classification.
Two other approaches that mainly focus on machine learning are described in [17] and [18], in which the authors perform the detection through neural network architectures.In the first paper, the authors especially focus on feature extraction, studying the most relevant attributes of text news.They identify different kinds of features: content features, such as the number of words and the frequency of characters, user features, based on the news readers and in particular on the users who have interacted with the news, and, finally, social features, which refer to the social connections of the users.All the features, considering text and news context, are then evaluated and compared using a Convolutional Neural Network (CNN) and a Long short-term memory (LSTM) that provide the news classification.In the second work, the authors propose a classification model for fake news detection based on linguistic features and automatic fact-checking.The model evaluates the news considering linguistic features, such as the number of words and sentences, and then compares them with mainstream verified articles; a deep learning algorithm is trained to learn the common patterns and produce the classification.
Furthermore, two different machine-learning techniques are illustrated in [19] and [20].In the first approach, the authors propose a fake news detector based on the analysis of term frequency and unique words.After this feature extraction process, a Naive Bayes Classifier (NBC) model is trained and then proceeds to the news classification.The system is evaluated considering the precision and the accuracy of classification.In the second one, the authors analyse the association between fake news and clickbait and how in general, the goal of fake news producers is to profit through clickbait.Clickbait lures users and raises curiosity with flashy ads or designed click links to increase revenues.In these terms, the authors propose a fake news detection model based on context analysis, e.g., collecting URLs commonly used for clickbait and linguistic features, such as the number of capitalised characters or exclamation marks.Moreover, a work based on well-known machine-learning techniques is illustrated in [21].The authors propose an automatic The last group of articles focuses on different approaches that do not consider machine learning techniques for fake news detection.In these terms, two works are illustrated in [22] and [23], in which the authors mainly concentrate on sentiment analysis.In the first work, a framework to encourage fact-checked content is proposed, and the authors examine active Twitter users, called guardians, who share validated information in order to correct fake content in online discussions and provide them with a URL-based fact-checking recommendation model to stimulate their engagement and reduce the negative effects of fake news.At first, the proposed model focuses on the detection of the guardians' users, then analyses the textual claims and recommends guardians' fact-checking URLs to the other users.In the second work, the authors propose a model to detect fake news using sentiment analysis as the main feature.The model combines the sentiment related to the text with style features, such as the number and frequency of words and statements, and performs the classification through different machine learning algorithms.
In summary, Table I shows a classification of the fake news detection models based on the analysed approaches.The classification is based on three metrics, i.e., text quality, fact-checking, and context analysis.In general, the features needed to measure the news quality can be divided into four main categories [24]: Quantity, which considers features such as the number of characters, the number of words and the number of sentences; Informality, which takes into account the fact that fake news often contains more mistakes than trustworthy ones, and therefore characteristics such as misspellings and typos are used as indicators of the authenticity of the news; Complexity, which is represented by parameters such as the average word length, the words per sentence and the average ratio of punctuation per sentence.In general, the higher the linguistic complexity of a text, the less likely it is a fake.Finally, the last category is depicted by the Diversity, which considers the percentage of different terms in the text, the occurrence of the words, and their spatial distance; deceptive texts are perceived to be limited in terms of vocabulary usage and usually make use of several redundant terms.Another classification metric is represented by Fact-Checking.Even if the text analysis is largely used for classification, other approaches are used to support it.To verify the correctness of the information, researchers propose systems to check the credibility of the news.The last metric concerns the analysis of the news context.It classifies news through the analysis of the website in which the news is published with features such as the Sentiment related to the news content or the presence of advertisements (Ads) as well.The solution proposed in this paper takes into consideration all kinds of news topics, and no user information is collected; for this reason, properties related to the news readers are out of the scope of our solution.

B. Trustworthiness on News Providers
The issue of trust in news providers has been gaining huge popularity in recent years, and the community is trying to find new approaches to study the news providers' trustworthiness [25].This is due to the fact that non-verified news media have exploited technological development to spread misleading or fraudulent news [26], [27].In general, trust in news providers is at historically low levels and new mechanisms to measure their trust are essential to overcome this age of uncertainty [28].
Below, we provide a brief background on the most accepted properties used in literature to evaluate the trust of a news provider.One of these is presented in [29], where authors introduce the concept of trustworthiness in news media as a relationship between a trustee, represented by the user or the actor who trusts, and a trustor, i.e., the news source that provides the news and receives the trust.The news media is then evaluated, and the trust is obtained through 4 dimensions: trust in the selectivity of topics, selectivity of facts, the accuracy of depictions, and journalistic assessments.Another two works that analyze the impact of trust in news media are illustrated in [30] and [31].The first paper proposes various measures of trust at different levels of analysis.The source is evaluated both generally and in each specific topic, taking into account its fairness and accuracy in distinguishing between facts and opinions.Then, in the same way, the paper judges the author of the news, the journalist, and the media outlet, according to their objectivity and subjectivity.In the second paper, the authors examine the news media trust in terms of credibility in a political topic for five different countries.The authors present a regression model that assesses news providers based on various features, including media attributes such as overall credibility and reading frequency, as well as demographic metrics of the news reader.
In these terms, we propose an automatic algorithm to evaluate the trust of news sources based on the most accredited measures of credibility.It makes use of novel parameters, which consider the expertise, the popularity of the news source, and its past behaviour.

III. INTRODUCTION TO THE PROPOSED SOLUTION
This section provides a detailed description of the behaviour of the entire system.All the functionalities described here are Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.presented to the final users through a plugin which interacts with the FAKE system.
The FAKE system is comprised of three entities, as shown in Fig. 1, which are available after an authentication phase: a fake news detector, a trust management model, and an encrypted database.The fake news detector is responsible for evaluating news.When a user requests news, the system initiates the evaluation process to provide feedback on the requested information.To this, the detector extracts important features from each news item and implements several algorithms to provide feedback.The purpose of this entity is then to store the feedback on the database for future uses.The trust management model is designed to evaluate news producers' websites: the model has access to the database in order to retrieve all the information regarding a specific news source, such as feedback or news topics.This information is aggregated to extract novel parameters, which are used to evaluate the reliability of the news source.Finally, the database has the important role of acting as an intermediary between the two previous entities, allowing to save and retrieve the evaluation of the news.To ensure the security of exchanged data, even in the face of potential data leaks, we adopt the Blowfish algorithm to encrypt the results of the evaluation process, which represents the cornerstone of the entire system.We have chosen this algorithm since the encryption is only used internally to the system and there is no need to send feedback data to third parties.However, we only encrypt feedback data, leaving non-sensitive data unencrypted.This approach reduces the computational and time costs associated with encryption, making it an effective means of protecting confidential data [32].
The plugin continuously monitors the web pages visited by the user.The activation of the fake news detector or the trust management model depends on the user's browsing behaviour.To illustrate the system's functionality, we provide two sequence diagrams in Fig. 2. Solid arrowheads indicate calls to system entities, while dashed lines represent reply messages.If the plugin detects news (Fig. 2(a)), it triggers the fake news detector.The detector evaluates the reliability of the news: to this, it first checks if the news has already been assessed and if relevant feedback has been stored in the database.In this case, the detector retrieves the feedback value and immediately displays it to the user.However, if no feedback regarding the news is found on the database, the detector starts analysing the news and extracts all the parameters of interest, such as the topics, URL, and source, as well as features related to the text, sentiment, and more.The detector then computes the news feedback based on the model explained in Section IV and stores the feedback, along with the associated information, in the database: feedback can then be retrieved by any of its parameters, such as the URL of the news, source, topics, timestamp, and so on, so the proposed feedback has multidimensional views.In order to keep our formulas as clean as possible, we will only address the feedback with the minimum notation needed to explain the model, but the reader should keep in mind that the other parameters are only hidden but always available.
Whenever a user is looking for news on a browser or a news producer's website (Fig. 2(b)), the plugin interacts with the trust management system.In order to compute the trustworthiness of the news producers, the trust model retrieves all the needed information from the database and shows the user the risk associated with every news website to provide the user with the best alternatives.The trust of a news source is evaluated by considering all the topics it has covered.This means that a source could be considered reliable for certain topics, but not for others.
Finally, we note how the collection of personal data takes place only when the user downloads the application, i.e., the plugin, to be integrated into their browser, and it will only be used for contractual purposes.During the browsing phase, no user data is collected, and the only information that the system collects is related to the web page that the user has visited anonymously.

IV. FEEDBACK EVALUATION MODEL
According to the presented scenario, in this Section, we propose our feedback evaluation model, which is responsible for assessing the news selected by users.In order to evaluate and assign feedback, the model considers several parameters, which can be classified into three different factors: message-based, fact-based and context-based parameters.The first factor refers to the style of the news and proposes to analyse the text's characteristics based on quantity, informality, complexity and diversity.The second factor considers the examination of facts comparing news claims with a large well-known pre-trained model or with news already evaluated.Finally, the context-based parameters consider the presence of ads on the web page and sentiment analysis, which evaluates the text in terms of sentiment and objectivity as well.Therefore, the feedback f i related to a news n i is computed as: where M i represents the contribution of the message-based, while F i and C i depict the fact-based and context-based parameters respectively.All these factors, namely M i , F i and C i , are in a range [−1, 1], while the weights α, β ∈ [0, 1] are selected to give more importance to a particular factor.The weights are selected so that their global sum is equal to 1 in order to normalize the feedback value in the continuous range [−1, 1], where the value  equals to −1 depicts news generated to harm someone or something, i.e., new articles created to spread disinformation [33], while the unitary feedback value corresponds to reliable information.Among the concept of information and disinformation, values of f i around zero indicate misinformation, i.e., false information shared without the intention to harm [34].There exist two zones of uncertainty where the classification is difficult.These zones represent the transition from disinformation to misinformation and misinformation to information.In order to resolve this uncertainty, we define a threshold T H so that all f i : f i ≤ |T H| are classified as misinformation.

A. Message-Based
The Message-based factor represents the first set of parameters related to the analysis of the news through their style which make use of the text quality to distinguish false statements from real ones.At first, the text news is pre-processed in order to clean the text by removing special characters and stopwords, making it ready to feed the text to our model.Then the model proceeds to the feature extraction step so that the writing quality can be measured based on features of quantity, informality, diversity, and complexity.In these terms, Table II summarises all the features necessary to analyse the message-based factor of news text.After the feature extraction phase, we make use of the XGBoost (Extreme Gradient Boosting) algorithm [35] to compute the message-based value M i ∈ [−1, 1].Concerning other classifier algorithms, XGBoost does not present issues with poorly cured datasets and allows us to find out which features are more dominant and important for classification.

B. Fact-Based
The Fact-based factor ensures the correctness of the news information and analyses its level of truth.It concerns two different contributions: the parameter F i,f , which evaluates the news by matching its claims with a pre-trained network, named FEVER, of verified claims, and the Fact Comparison parameter F i,c , which compares the news article with similar ones and uses this comparison for its evaluation.
1) FEVER: This parameter is responsible for the examination of facts by matching the text news with a pre-trained network based on Wikipedia claims, namely FEVER [36].The network is able to classify a specific claim into three categories: supports, refutes, and not enough information.The first category refers to approved claims, i.e., the algorithm has found a correlation with the Wikipedia dataset.The second one considers false claims, which means there is evidence in the dataset that prove the claim is false, while The final category indicates that the network is unable to find any evidence regarding the reliability of the claim in question.
At first, the news text is processed and split into its different claims; we then define a set of claims C i = {c p,i } associated with news n i .The generic claim c p,i can have two states: 1 for true claims that align with the FEVER dataset, and 0 for false or insufficient information claims.Therefore, the number of supported claims V i is expressed as follows: However, the number of reliable claims is highly dependent on the length L i of the news, so naturally, longer news has more verified claims w.r.t.shorter news even if the news itself is not necessarily true.To this, we normalise the number of verified claims with the news length, i.e., V i /L i , and use this ratio, which values are in the interval [0, 1], as an indicator of the reliability Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply. of the news.As shown in Fig. 3, the number of verified claims, obtained from the Kaggle dataset, which will be explained in detail in Section VI, grows much slower when compared to the length of the news, so we expect that fake news will have a higher value of V i /L i w.r.t.real news.The FEVER parameter F i,f is then computed as: so that F i,f ∈ [−0.5, 0.5] to account for the incompleteness of the FEVER dataset.
2) Fact Comparison: This parameter evaluates the accuracy of the facts reported in the news by comparing them to reliable and previously evaluated news, which are selected using a similarity algorithm.Since ground-truth information is unavailable for every news item, this approach provides a means for assessing news precision.We make use of the similarity approach presented in [37] in order to find comparable news articles that were already evaluated and stored in the database.At first, the considered news is processed, and a bag of words is generated through the combination of several embedding algorithms, such as Word2vec or GloVe.Then, the model has access to the database and retrieves similar already evaluated news with a higher value of feedback.Therefore, the news n i is associated with a set of similar news where f th represents the threshold beyond which the news is classified not only as information but also as reliable, A i,j ∈ [0, 1] depicts the similarity coefficient between two news n i and n j , and finally A min is the minimum acceptable value of similarity.In specific, values of A i,j close to 1 refer to highly similar news, while 0 refers to completely different ones.For each similar news, the model associates its specific set of words, and so the similarity algorithm, namely the cosine similarity, is adopted to compare them and find the closest news.The fact comparison parameter F i,c ∈ [−1, 1] is then calculated only if there is similar news, i.e., if |D i | > 0, as follows: where f j represents the feedback of similar news n j .
Finally, the fact-based factor F i ∈ [−1, 1] for a news n i is computed as follows: where if the number of similar news is equal to 0, so |D i | = 0, the factor is calculated based on the FEVER parameter; otherwise, the fact comparison parameter is used to evaluate the credibility of the news claims.

C. Context-Based
The context-based factor takes care of all the parameters that are not directly related to the news but involve its context.In specific, it concerns the presence of ads on the web page C i,a and the sentiment analysis C i,s .These two parameters are described below: 1) Advertisements: Low-credibility news sites usually make use of ads to gain significant revenue by attracting users [38].For this reason, the system proposes to detect the common pattern of ads and take advantage of the correlation between fake websites and the number of ads as follows: with . N i,a depicts the number of ads detected on the news web page, while N i,a represents the maximum value after which every number of ads corresponds to the lowest value of the score, that is −1.
2) Sentiment Analysis: This parameter has the important role of detecting discrepancies between the sentiment related to the news text and its title, and it analyses the global text objectivity.The sentiment analysis determines if the information of the two components, text and title, is expressed in a positive, neutral, or negative way [39], and, in addition, it depicts the sentiment polarity, i.e., the strength of negative or positive sentiments [40].Usually, fake content mixes different information with positive or negative feelings to mislead readers.Moreover, subjective language is commonly exploited by fake providers that focus on personal interpretation rather than factual data from an objective point of view [41].The proposed model makes use of the sentiment algorithm suggested in [42] in order to analyse the sentiment related to the news title S i,title and the news text S i,text ; moreover, it takes advantage of a well-known sentiment analyser illustrated in [43] to evaluate the text objectivity.Therefore, our sentiment parameter presents two contributions: the first refers to the dissimilarities between text and title sentiment, which indicate if the title is coherent with the reported news and is not only a clickbait; meanwhile, the second one evaluates the global objectivity of the news text.The overall sentiment analysis factor C i,s ∈ [−1, 1] is then calculated as follows: where σ ∈ [0, 1] depicts the weight selected to give more importance to a specific parameter and O i ∈ [0, 1] represents the level of objectivity; the value of 0 refers to a very subjective text, while Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
1 to a completely objective point of view.Moreover, the sentiment distance parameter ΔS i = |S i,title − S i,text | measures the difference between the sentiments expressed in the news title and text, where ΔS i ∈ [0] [1] and S i,title , S i,text ∈ [0] [1].
Finally, the context-based factor C i ∈ [−1, 1] for a news article n i is expressed as follows: where ρ ∈ [0, 1] is selected to give more importance to a specific contribution.Values of C i close to −1 indicate fake content, while positive scores near 1 refer to real ones.

V. TRUST MODEL
According to the scenario presented in Section III, the trust model is designed to evaluate the news producers and estimate their credibility.Therefore, the model accesses the database to retrieve all relevant information about a specific news source, including news feedback and topic.When the system evaluates a source s on a specific topic t, it calculates the trust value using the following formula: All these addends are in the range [0, 1] and the weights are selected to give more importance to a specific factor so that their global sum is equal to 1, in order to normalize the trust value in the interval [0, 1].The trust value T t s is evaluated based on three novel factors: the Expertise E t s , the Coherence H t s and the Goodwill G t s .As expressed in Section III, the feedback related to specific news has multidimensional views and can be described using f t s,i , which is related to several parameters, including the news source s, the topic t, and the news item i.In Section IV, we used a simplified definition of f i to provide a general description and illustrate its composition in detail.Below, we continue using the analysed notation and express the feedback as f i for better reading.As a consequence, the definition of topic t and source s is omitted in all the parameters that are related to them, e.g., the trust factors described as follows: the Expertise E, the Coherence H and the Goodwill G.

A. Expertise
The first factor quantifies how the source is well-informed on a specific topic.In specific, the expertise factor E is evaluated based on two parameters: the Topic Importance E m , which analyses the impact of the topic t in all the topics tackled by the provider s, and the Writing Competence E c , which considers proficiency in writing news of the source on the evaluated topic.
1) Topic Importance: The first parameter measures the expertise of the source based on the number of news published on the evaluated topic and discriminates specialised and qualified providers from general ones.The Topic Importance E m ∈ [0, 1] is then computed as follows: where N represents the number of news belonging to the source s on the topic t, while ( t N t s ) depicts the total number of published news by that source.
2) Writing Competence: The second parameter refers to the writing style computed as the average of the message-based evaluations, calculated according to Section IV-A.The Writing Competence E c is then computed as: where represents the value of message-based factor of the news n i published by the considered source s on the topic t.
Finally, the global Expertise E ∈ [0, 1] is expressed as: where values of E close to 0 refer to sources with a low level of expertise, while scores close to 1 indicate very capable and expert providers.Moreover, the weight τ is used to give more importance to a specific contribution.Specifically, the weight gives more influence to the topic importance parameter with high numbers of published news, while the expertise on a topic is not relevant with only a few pieces of published news.In these terms, τ ∈ [0, 1] is expressed as follows: where the weights ω ∈ (0, 1] and ψ ∈ [1, inf) are used to set the asymptotic value of the weight and to configure its speed at the variation of N; a more detailed explanation of these weights will be provided in Section VI-C.

B. Relevance and Goodwill
These last two factors are used to study the dissemination of the source's news among the users and to take advantage of the social impact of fake news in the detection mechanism.Several research models demonstrate the risk of fake content spreading in social networks and how the perceived information quality is influenced by the intention to re-share information [44].In these terms, we propose the Relevance factor R i ∈ (0, 1] for a generic news article n i to determine how frequently a news source's articles are consulted and to weigh its evaluation accordingly; the factor is expressed as: where N r,i represents the number of times the news n i is requested by users, N r defines the total number of news requests for the source s within the topic t, and ( N i=1 N r,i = N r ).The news feedback f i is then weighted through the Relevance factor and the global Goodwill factor G ∈ [0, 1] is computed as follows: Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
where R i and f i represent the Relevance factor and the feedback related to the news n i , respectively.

C. Coherence
The Goodwill factor may not be effective in responding to sudden changes in a source's behaviour, as it happens for dynamic attacks such as On-Off Attacks (OOA) [45] and Whitewashing Attack (WA) [46].Indeed, sources that implement these attacks periodically change their behaviour, e.g., by alternatively being benevolent (ON) and malevolent (OFF) or by registering again as news providers with a different identity.To overcome these attacks, this factor evaluates the recent behaviour of a source considering a small temporal window, which makes use of the last N last news evaluated for source s on topic t.The Coherence factor H ∈ [0, 1] is then computed as follows: where f i depicts the feedback of the news n i and N last represents the dimension of the temporal windows of considered feedbacks.To give more relevance to the latest feedback, with regard to the oldest one, the weight w z of each feedback follows a geometric distribution with parameter p: where to maintain the parameter range [0, 1], we introduce the term ξ res , which accounts for all the residual weights of the distribution due to the transactions older than N last .Therefore, ξ res is computed as follows:

A. Simulation Setup
In order to test the proposed system, i.e., both the fake news detector and the trust management algorithm, we need a large dataset of news.To this, we make use of the dataset made available by Kaggle at [47], which contains a total of 20387 news.Kaggle is a platform that hosts data science competitions and organizes tournaments for recruitment and academic research.In detail, the dataset is in a Comma-Separated Values (CSV) format and consists of four attributes: title, author, text, and label.The title is the headline of the published news; the author represents the journalist who wrote the news; the text is the body content that describes in detail the news story and, finally, the label attribute defines, through a binary classification of zero or one, whether the news is false or real, respectively.
Given the large size of the dataset, we consider that out of all the articles, 90% of them are used for training the message-based factor, whereas the remaining 10% was used to test its performance.However, a preprocessing phase was required to manage the data for our purposes properly; first, we adjusted the ground truth labels so that fake news is labelled with -1, while genuine news articles have a label 1.Second, the system required the news source, which is used by the trust evaluation model: thus, the author field was discarded and replaced by the news article URL.Another effort for retrieving the necessary attributes to the overall system, which was not comprised of the original dataset, was the main topic covered by the news.To retrieve this information, we implemented a deep learning approach to detect it automatically.The purpose of this network is to classify each news article into one of 10 possible topics: Arts & Culture, Business & Economy, Crime & Security, Entertainment & Celebrity, Health & Education, Politics, Science, Sports, Tech, and finally, Weird.This classification was obtained using a dataset of 60,000 news items gathered from various news producers' websites with web scraping techniques.In particular, the dataset consists of two attributes: news' text and category.The news text is cleaned and embedded in order to train the network.Therefore, a percentage of 80-20% was used to divide the dataset among training and testing using the news embeddings as the feature vector.The results were satisfactory and precise for our purpose: we achieved a test accuracy of 91%.
Finally, the dataset only classifies news as fake or real, so to test our detector algorithm, we consider that news with feedback lower than 0 is fake, and real otherwise.Table III shows the optimal configuration of the simulation parameters for the proposed system and the different weights used for the model.More details on the selection of these parameters are given in the following Sections.

B. Simulation Results for the Feedback Model
The feedback model aims to evaluate news from various perspectives and aggregate them in order to provide a feedback score that represents the reliability of a specific news article.Therefore, the feedback score f i related to news n i is a proper combination of metrics based on the message, fact, and context parameters, in which values near -1 mean n i is likely fake news, i.e., it has the temperament of disinformation.On the other hand, values around 1 suggest that the feedback model detects news that provides information.

1) Message-Based:
The first set of simulations aims to validate the performance of the XGBoost algorithm and to compare it with other four machine learning algorithms, Naive Bayes Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.(NB) [48], Support Vector Machine (SVM) [49], K-nearest neighbours (KNN) [50] and Random Forest (RF) [51].In order to evaluate the classifiers' performance, we used two well-known metrics: the Receiver Operating Characteristic (ROC) curve and the Area Under the Curve (AUC).The ROC curve measures the true and false positive rates at different classification thresholds.Alternately, the AUC illustrates how the model is accurate in achieving the classification through the output classes, e.g., AUC = 1 means that the system is 100% accurate.A comparison of the proposed classifiers is shown in Fig. 4, where the XGBoost and RF emerge as the most appropriate choice.Among these two best-performing algorithms, we decided to use XGBoost, which presents the best results in terms of computational speed in our simulations, with a total training time of less than a minute.With this setup, the message-based factor reached a 92% accuracy, with 91% concerning fake and 93% for real news.
2) Fact-Based: The focus of the following set of simulations concerns the testing of the fact-based factor, i.e., the combination of the contributions from FEVER, which has been tested to retrieve the number of supported claims, and from the fact comparison with already evaluated news.
We decide to consider the number of supported claims for news due to the study of [52], which reveals how FEVER improves its accuracy in detecting only supported claims.At this point, the Stanford Parser tool [53] was used to decompose each news article into triplets, i.e., subject, verb, and object, and to remove unnecessary parts of the speech, so as to provide a simplified input for FEVER.Once the data was fitted properly, FEVER was employed to compute the number of supported claims V i for news n i .Fig. 5 shows the FEVER contribution F i,f by displaying the percentage of real news within each bar with different colour shades.For a clearer view, each bar reflects an aggregation of 400 news.As expected, high values of V i /L i indicate a high concentration of fake news, whereas low values imply a prevalence of real news.Additionally, the more the FEVER parameter is close to the boundaries, the more its accuracy increases.Finally, the FEVER parameter reaches an overall accuracy of 70%, computed by assessing fake news if F i,f < 0 and real news if F i,f ≥ 0.
The second contribution of the fact-based module is the fact comparison, which employs the concept of similarity.As explained in Section IV-B2, the concept of similarity is exploited in order to achieve a comparison with previously analysed news.
In detail, we operate on two parameters: the similarity threshold A min and the minimum feedback threshold f th , to consider only a small group of similar and reliable news already evaluated.
Fig. 6 shows the fact-comparison accuracy scores (compared with the steady trend of FEVER) by varying the similarity and the feedback thresholds.The highest accuracy value can be obtained using stringent thresholds, i.e., by selecting news with a 90% of similarity and feedback of at least 0.8; however, these thresholds are too demanding, and only the 5% of news could be evaluated with them.By loosening the thresholds slightly, i.e., by setting A min = 0.87, we were able to include the 23% of news and obtain an accuracy better than FEVER.For all the other news, which can not be evaluated with the fact comparison, the system returns the value computed by the FEVER contribution.
3) Context-Based: The set of simulations on the context is divided into two categories: the analysis of the ads available on the news web page and the sentiment analysis.Fig. 7 depicts the impact of advertisements on the news classification; indeed, real news exhibit only one or two ads and it is clear how after Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE IV INVESTIGATION FOR THE SENTIMENT ANALYSIS TABLE V SYSTEM'S FACTORS ACCURACY FOR THE TWO ANALYZED DATASETS
a certain number of ads, denoted by N i,a in our model, greater than 2, the presence of real news drastically decreases.This allows the system to provide a completely negative feedback score, i.e., C i,a = −1, if N i,a > 2. Furthermore, we analyse the importance of sentiment analysis.Table IV shows the mean and variance of the sentiment for the title and news text as well as their difference ΔS, and the mean and the variance of the text objectivity.The results demonstrated that fake news tends to have more divergent values, with a greater average and variance.The system takes advantage of these differences and increases the accuracy of fake news detection so that the final accuracy of the context-based factor is 76%.
4) Feedback Score Overall: This Section concerns the final aggregation of the three factors analysed above: message-based, fact-based and context-based.The message-based contribution is the most accurate, so the weights are chosen to provide more importance to this factor.The best results are obtained by setting α = 0.7 and β = 0.15, which resulted in the same accuracy as the message-based factor alone.Indeed, giving more weight to the fact-based or context-based factor resulted in lower accuracy.However, we decided to investigate each factor's ability to detect fake news, so we analysed the accuracy of each factor at different intervals employing an additional dataset to generalise the weights' choice.The new dataset has been collected by Ahmed et al. [54] with almost 40,000 news articles.The results are reported in Table V.From this analysis, we can notice that the message-based factor is less accurate in those intervals, which are close to zero.In these intervals, fake news behaves similarly to factual news in terms of detailed information and writing quality.From these considerations, we use the term "misinformation" to describe these areas of significant ambiguity.The new term does not necessarily fit into the solution's metrics, although the term "misinformation" follows the literature and explains this area of uncertainty.However, in terms of accuracy, we distinguish only fake and real news by checking whether the feedback is respectively lower or greater than zero.Hence, due to the uncertainty intervals of the message-based factor, the weights can be adjusted to exploit its low accuracy.This means that α and β have no constant values but rather their value changes to take advantage of the strong point of each factor to improve the overall accuracy.In order to allow only the factors with good accuracy to provide a contribution in the final aggregation, the weight for all factors with accuracy less than 0.5 is set to 0. We, therefore, need to re-scale the accuracy from [0.5, 1] to [0, 1].To this, we are interested in assigning greater weight to the factors that have higher accuracy values, so we have adopted a non-linear transformation and, in particular, a square function with the vertex in (0.5, 0) and passing through the point (1,1).Finally, we normalise the obtained accuracies so that their sum is equal to 1, to obtain the weights for the three factors.We then tested these weights for the two databases proposed, and we achieved a 94% accuracy for both the Kaggle dataset and the dataset proposed by Ahmed et al.Finally, we remark that even though the overall accuracy achieved is only 2% higher than message-based accuracy, the fact and context-based accuracy provide a 25% increase in the accuracy in terms of remaining errors.
In conclusion, a comparison with previously studied authors who employed the same Kaggle dataset as our solution is provided.Specifically, we tested our algorithm on the abovementioned dataset by keeping the exact same number of news used by related works in order to have a correct comparison among them.Drif et al. [17], through multiple approaches such as content-based, user-based and social-based, achieved the lowest accuracy of 72.5%.Another significant work is covered by Ahmed et al. [16], in which they tested two methods: the first one with an LR-unigram and the second one with LR-LSVM obtaining an accuracy of 89% and 92% respectively.Finally, another similar approach through the same dataset was employed by [19], using a Naïve Bayes classifier.They split the dataset into 70% for training and 30% for the test part and obtained 92% accuracy, which is perfectly comparable with our message-based performance.Table VI summarises the performance of these works and compares them with our solution.

C. Simulation Results for the Trust Model
We evaluate the performance of the proposed trust model by analysing the trust value.A fine-tuning of the weights related to the three metrics, i.e., Expertise, Goodwill, and Coherence was necessary.The most relevant results have been achieved by setting γ = 0.25 and δ = 0.25.Fig. 8 depicts the trust value  with these last parameters when a source performs three distinct behaviours.In general, media outlets could modify their behaviours for several reasons; this happens for the competitive nature of journalism, which can lead to reporters feeling pressured and publishing news as soon as possible without verifying its authenticity [55].To test it, we adopt three different behaviours: a benevolent one, in which the news provides verified and real information, and two dynamic ones, aimed at testing the robustness of our system to news media that modify their behaviours after publishing either real or fake news.In the first dynamic behaviour, the source builds its reputation with 50 trustworthy news and then starts providing 100 fake news, while in the second one, the source begins publishing fake news and then tries to improve its reputation with 100 reliable news after having provided 50 fake news.The simulations show how the algorithm is able to quickly adapt to the changes thanks to the Coherence factor.
The following results focus on a better understanding of how the trust model avoids the spreading of fake news.To this, we have organized simulations with synthetic data: first, an experiment was conducted to build an attack for relevant news.The first group of 20 real news is induced to bring the algorithm to convergence.Then, a single piece of fake news is requested 20 times, followed by 20 different fake news requests; finally, 70 real news are requested to see how the system reacts.
Fig. 9 depicts the system's response to these attacks in two cases: with and without the relevance factor.It is clear that once the source publishes the first 20 real news, the trend is the same for both algorithms.The discrepancies occur when the same fake news is required multiple times, making the system comprehend that the news article is significant.The trust score rapidly decreases when the algorithm detects news requested several times, and so it penalizes the source.In contrast, when relevance is not employed, the news provider is punished only for the single fake news, and the trust score is steady until the source publishes 20 more different fake news.Moreover, at this point, we can notice that also the trust value of the source with relevance changes behaviour due to the different penalization: indeed, at first, the negative contribution was due to the multiple requests of the same fake news already stored in the database.However, the curve decreases rapidly due to the bigger impact of new fake news evaluation.In addition, during the burst of 70 real news, both curves are able to regain a part of their trust score.Another interesting result can be noticed during the rising edge: two points present abrupt changes due to the Coherence factor, employing a short temporal window N last to assess the recent news.When the temporal windows N last contain positive feedback scores, the trust value increases more rapidly.Another significant achieved result is described in Table VII.The simulation is based on the real dataset that includes sources that primarily write on the topics of Business & Economy (B&E) and Politics.The table shows the quantity of fake and true news stories for each source and topic.Furthermore, the average feedback is reported, followed by the source's trust score on that specific subject.A considerable result is given from the fact that, although the source "Breitbart" has a lower percentage of news labelled as fake by our fake news detector than the source "NYTimes", the latter receives a higher score.This is because our system not only detects fake news but assesses its quality.Indeed, a news story with a feedback score close to 1 is well-written due to the design of the feedback evaluation score, which contains continuous values and is provided by the high impact of the message-based.
Finally, we want to understand how the accuracy of detecting fake news affects the trust model evaluations.Fig. 10 represents a reliable source that publishes news.The ground truth case (black line) has been highlighted, in which the fake news detector is 100% accurate.In this case, the feedback evaluation concerns a discrete rate where fake news has a feedback score of f i = −1, while f i = 1 is assigned to real news.In contrast, the red line represents the trust score obtained by the feedback computed by the proposed fake detector.In addition, we want to analyze the results at varying the error percentage in the feedback evaluation.The other curves show how the trust model reacts with a 10, 20, and 30% error, respectively, and how it can follow the real trend.Although the feedback model is not 100% accurate, the trust model manages the errors quite well by following the ideal trend until an error of 20%.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

D. Simulation Results for the Entire System
This last section concerns the functioning of the entire system in a real environment.The system is tested by simulating the activity of multiple users looking for news on a web browser for websites belonging to three well-known media sources, i.e., BBC, Breitbart, and USA Today.In these terms, Table VIII illustrates the specific of the machines used for the following simulations, where the Client indicates the machine used by users and in which the plugin is installed.The Server is running the FAKE system, responsible for all the evaluations.We evaluate the performance of the system by testing 100 simultaneous users, in which all the plugins (clients) detect news and send them to the server for evaluation.The results are analysed in terms of the time processing necessary for the server in order to perform evaluations.In this regard, the system is able to evaluate news in less than an average of 2 seconds and is able to provide the evaluations for news already evaluated in less than 0.2 seconds.Furthermore, we analyse the impact of the plugin on browser performance by measuring the Average Time Execution, which represents the average time needed for displaying the evaluations in browsers, and the Average Memory Usage, which is the average amount of memory, usually expressed in bytes, required to load the data.Table IX illustrates the average impact of the plugin by analysing the browser without and with the usage of the plugin.Simulations illustrate the low effects of the plugin in terms of time and memory usage.[56] with traditional databases.Blockchain can be implemented as an alternative to a secure database, and it has been used to save all feedback and source evaluations [57], [58].In recent years, the Blockchain has gained massive popularity in many research areas, with several approaches proposed for using it to combat fake news and provide a transparent and secure environment.
Blockchain can be described as a digital ledger of transactions, duplicated and distributed across a decentralised ecosystem, enabling trust in peer-to-peer networks without the presence of certification authority.In contrast to centralised systems, Blockchain has overcome several security weaknesses, such as being tampered with by malicious actors.Furthermore, due to the distributed ledger maintained using the distributed consensus algorithm, the Blockchain enables the involved actors to avoid third-party trust in interactions, guaranteeing traceability and security [59].More abstractly, Blockchain can be seen as an ordered list of blocks, where each block represents a register that keeps information and transactions, and it is linked to the previous one in chronological order.The chain starts with the genesis block, representing the first block.All the information in all blocks is encrypted, and a consensus algorithm ensures reliability for new nodes in the network.For this comparison, we have implemented our system, making use of both an encrypted database, as described in the rest of the paper, and the Ethereum Blockchain proposed by Alastria [60].In the latter case, each news evaluation provided by the detector is stored in the Blockchain, and a new block is created; this process guarantees that any modifications can not be achieved without changing all the previous blocks.Moreover, the Blockchain provides real-time responses to the request of news that are already evaluated and are necessary for the credibility measure of the news providers.
In an Ethereum Blockchain, the smart contracts allow the users to execute a script on a Blockchain network in a verifiable, way and so create and retrieve transactions using private functions.An Ethereum node can follow the instructions in a smart contract and execute them from a valid account.The contract determines the structure of the data that can be stored in the Blockchain and the functions with which the nodes operate within the Blockchain.In this solution, we wrote a smart contract making use of the Solidity programming language.Furthermore, in order to compare the Blockchain with the encrypted database, Table X shows a set of simulations for both actions, i.e., write and read.We evaluate the performance by analysing two metrics: latency, which specifies the time taken to read or write data, and throughput, which represents the amount of information that can be written or read in a given amount of time, typically measured in bits per second (bps).The Table illustrates that the encrypted database outperforms the Blockchain in both metrics, and it exhibits better performance in terms of processing time, making it the preferred solution for real-time applications.Moreover, one ongoing challenge in Ethereum is scalability, given that the platform currently processes approximately 500,000 transactions per day and has a maximum capacity of about 15 transactions per second [61].However, in scenarios where trust, robustness, and data provenance are the system's top priorities, Blockchain is the best solution and overcomes several security weaknesses, such as tampering by attackers [62].

VIII. CONCLUSION
In this article, we have proposed a detection algorithm for text-written news, classifying articles as real or fake according to several parameters related to the text style and the news context.Moreover, we have developed a trust and reputation system so that it is possible for users to understand which news providers that can lead to successful collaboration, i.e., that can provide reliable news.The paper also presents a section to compare traditional databases with a Blockchain solution based on the Ethereum smart contract.Finally, we tested the whole system using a Kaggle dataset containing several articles from various domains.

Fig. 2 .
Fig. 2. Sequence diagrams for the fake news detector and the trust management model.

Fig. 3 .
Fig. 3. Analysis of the number of supported claims and news length for the Kaggle dataset.

Fig. 9 .
Fig. 9. Trust score with and without the relevance algorithm.

Fig. 10 .
Fig. 10.Trust model performance through different fake news detector accuracy systems.

TABLE II MESSAGE
-BASED FACTOR FEATURES

TABLE VII TRUST
AND AVERAGE FEEDBACK SCORES RELATED TO THE MOST EVIDENT SOURCES