A Comparative Study of Bayesian and Dempster-Shafer Fusion on Image Forgery Detection

With the advent of digital imaging, it has become fairly easy to modify the content of an image in many different ways while leaving no obvious visual clue. This has further challenged many existing image forensic techniques. The techniques which perform well with one specific kind of forgeries still suffer from strong limitations when dealing with realistic tampered images. Therefore, an effective strategy for tampering detection and localization requires the application of fusion technique. Although there have been extensive researches on fusion technique on different fields, there has never been a systematic study about fusion technique in image forensic domain. In this paper, we provide a thorough review on the state-of-the-art of fusion methods applied in tampering image detection and localization domain. We then present a practical comparison of two popular fusion techniques: Bayesian and Dempster-Shafer theory (DST) based fusion. The comparison relies on two applications which leverage the two aforementioned fusion techniques. In the first case, aggregating the decision maps of two forensic approaches: Photo Response Non Uniformity (PRNU) and statistical features based approaches has improved the forgery detection performance on saturated and dark regions of images. In the second case, integrating the decision maps of the forensic approach using demosaicing artifacts and the forensic approach using SIFT descriptors and local color dissimilarity maps has enhanced the detection performance on both copy-moved and copy-pasted forgeries images. Experiments show that the DST based fusion performs better in the first case while the Markov Random Field (MRF) based fusion performs better in the second case. It can be concluded that each technique has its own advantages and the best choice depends on each situation and users’ requirements.

sources. Hence fusion becomes a significant technique to 96 improve the performance in forgery images detection and 97 localization. However, how to select the different sources of 98 information to be combined and which method to fuse to 99 obtain a good performance in image forgery detection are 100 not evident tasks. To the best of our knowledge, there has 101 never had a systematic study before on the fusion technique 102 applied in forgery image detection and localization. In this 103 paper, we perform a review on information fusion in forgery 104 image detection and localization. We hope to help students 105 and researchers to have an overview about how to apply infor-106 mation fusion in forgery image detection. We then make a 107 practical comparison of Bayesian and Dempster Shafer The-108 ory fusion techniques applied in localizing forgery images. 109 This paper is an extension of [34] in which we present 110 a systematic review on information fusion in forgery image 111 detection and localization. We also propose a fusion frame-112 work to improve the forgery detection and localization by 113 integrating the decision maps of the forensic algorithms 114 which detect copy-paste and copy-move tampering. More 115 importantly, a comparative study of two fusion techniques 116 DST and Bayesian is presented. And finally, the experiment 117 results are tested on more datasets comparing to [34]. 118 The remainder of this paper is organized as follows. 119 Section II gives a brief definition of information fusion and 120 discussion on what and when to fuse. In Section III, we review 121 fusion methods which have been applied in forgery image 122 detection and localization. We then make a practical com-123 parison of Bayesian and Dempster Shafer Theory fusion 124 techniques applied in localizing forgery images in Section IV. 125 Finally Section V concludes our work. 127 In this section, we first define what information fusion is, then 128 discuss what source of information to be fused and when to 129 fuse in forgery image detection and localization problem. There have been existed many definitions of data fusion. 132 The authors in [15] gave a general definition of information 133 fusion as ''the science of combining measurements, signals, 134 or observations from different sources to obtain a result that 135 is in some sense better than what could have been achieved 136 without this combination.'' Due to this advantages, fusion 137 is a crucial topic in many scientific fields including sen-138 sors fusion, data fusion in internet of things [ [22], fusion in steganalysis [23] and fusion 141 in digital image forensics [24], [25], [26], [27]. jpeg. This causes the double quantization traces [28], 149 [29], [30], [31]. If  There are two main categories of fusion methods in forgery 207 image detection and localization problem due to its charac-208 teristics. Firstly, the decision outputs of forensic algorithms 209 are usually unreliable and imprecise because of limited tech-210 nical algorithm or particular characteristics of the considered 211 images (e.g. type of compression or saturated regions). There-212 fore, the information sources to be fused such as traces and 213 decision maps are often imprecise and uncertain. There are 214 then approaches capable of representing specific aspects of 215 imperfect data such as methods based on probability theory, 216 Dempster-Shafer evidence theory and fuzzy set theory, etc. 217 Secondly, the image forgery detection and localization can be 218 seen as a classification problem in which it outputs the label 219 (e.g. tampered or not tampered) or even a label probability of 220 each pixel in the considered image. Hence methods of fusing 221 multiple classifiers are also studied to improve the robustness 222 in detecting and localizing forgery images. In the follow-223 ing, we discuss four fusion methods including rule-based 224 fusion methods, probability-based methods, evidence reason-225 ing methods, classification based methods.

227
The rule-based fusion method includes a variety of basic 228 rules of combining such as linear weighted fusion (sum and 229 product), MAX, MIN, AND, OR. In [36] and [37] the authors 230 fuse the output maps of three forensic tools, based on sensor 231 noise, machine-learning and block-matching, respectively. 232 A decision fusion strategy is then implemented using the 233 simple rule AND, based on suitable reliability indexes asso-234 ciated with the binary masks. In [38], the authors proposed to 235 fuse three detectors which are PRNU based approach, Patch 236 Match based approach and Near-Duplicate based approach. 237 The tampering maps are merged with the AND operator, 238 according to a confidence value obtained evaluating the maps 239 on a training set of tampered images whose ground truth 240 tampering mask is known.

242
Probability-based methods rely on the probability distribution 243 which is defined based on the Kolmogorov axioms to express 244 data uncertainty. Among those, Bayesian fusion which lies 245 the Bayes estimator is one of the most powerful fusion 246 methodologies, especially for the fusion of heterogeneous 247 information sources. In fusion problem applied in detecting 248 and localizing tampering images, we are usually interested in 249 combining information such as traces, features, decision out-250 puts, etc. of several quantities of interest Z = {Z 1 , . . . , Z n }. 251 It is assumed that the information of each quantity of interest 252 In Bayesian fusion approach, 253 it is of our interest to compute the quantity P (Z | d 1 , . . . , d n ). 254 In the following, we discuss two approaches related to 255 image detection and localization problem.

303
Evidence reasoning methods include the method based 304 Dempster-Shafer (DS) theory, which is a mathematical theory 305 for modeling uncertain and combining evidence from dif-306 ferent sources to arrive at a degree of belief. Different from 307 Bayesian method, DS theory deals with measures of ''belief'' 308 which may not obey the classical probability axioms to repre-309 sent uncertain knowledge. It assigns mass function to repre-310 sent distribution of belief thereby not requiring to specify the 311 prior probability in advance. However, it does require mass 312 values to be assigned in a meaningful way to the elements of 313 the system.

314
In [27], the authors proposed a fusion framework based 315 DS theory to combine the output of several forensic tools 316 at measurement level thereby permitting to exploit as much 317 information about the tool reliability and about the compati-318 bility between the traces of tampering. More precisely, three 319 combinations are carried out hierarchically in their frame-320 work including incorporation of the output of each forensic 321 tool with its reliability, combination of different tools looking 322 for the same tampering traces and combination of different 323 traces. Each combination is merged by using Dempster's 324 rule and Basic Belief Assignment (BBA) is redefined on 325 the same frame using marginalization and vacuous extension 326 before being combined. The final decision is then made by 327 comparing the two belief values of two sets: T is the union of 328 all propositions in which at least one trace is detected and N 329 is the single proposition in which none of the traces is found. 330 These belief values Bel(T ) and Bel(N ) are calculated over 331 the BBA of the final mass function. A region is decided to be 332 tampered when Bel(T ) > Bel(N ).

333
In [40], the authors also proposed a fusion framework 334 based DS theory, yet, they not only fuse output of different 335 forensic tools but also integrate several background informa-336 tion into the framework such as tool-based information, trace-337 based information and semantic-based information. Taking 338 into account these side information which influences the 339 reliability of the forensic tools, the forensic performance is 340 enhanced. Particularly, some local properties of the image 341 such as saturated or textured regions affect accuracy of the 342 forgery localization maps. Thus the values of output map are 343 adjusted by mapping this local background information to a 344 BBA on the frame of the considered trace by using the method 345 proposed in [41]. Moreover, the global background affects 346 the output map when the estimated statistical model of the 347 tampered pixels and that of the original pixels are not well 348 separated. They then model the global information by defin-349 ing a new BBA. In addition, the compatibility relationships 350 between traces are modeled as a BBA using Dempster's rule. 351 Finally, the fused map is refined by exploiting the content of 352 the analyzed image.

354
Classification-based methods include fuzzy logic based on 355 theory of fuzzy set and algorithms using machine learn-356 ing such as K-Nearest Neighbor (KNN), Support Vector 357 Machines (SVM) and Naive Bayes (NB), etc.

358
The image forgery detection and localization can be seen 359 as a classification problem in which it outputs the label 360 VOLUME 10, 2022    The traditional classifier fusion approaches did not con-405 sider the conditional and spatial dependence of tampered 406 pixels with respect to their neighborhood pixels. The authors 407 in [49] solve this problem by proposing the Behavior 408 Knowledge Space representation fusion to integrate two best 409 approaches in the copy move detection: block-based and 410 points of interest detection methods.

411
The authors [33] integrated the tampering maps of statis-412 tical feature-based detector (M Fea ) and copy-move forgery 413 detector (M PM ). They first projected the score of original 414 and tampered pixels of the training forgery images on the 415 M Fea − M PM plane, then manually designed a decision curve 416 with fewer parameters which is effective and faster compar-417 ing to linear and non-linear classifiers such as SVM. The 418 experimental results show that this fusion strategy gives better 419 performance than the fusion based DRF and fusion based 420 supervised learning.

433
In this section, we compare the two dominate fusion tech-434 niques for multi-algorithms: Bayesian fusion and DST fusion 435 at decision-level fusions. We study the Bayesian fusion 436 as an optimization approach as mentioned in the sub-437 section III-B2 which corresponds to energy minimization 438 problem.

454
Ignoring the constant term P (m), the problem can be rewrit-455 ten as: Assume the independence between m i , we have Assume the independence between methods, we have: The prior is usually assumed the smoothness of neighboring Clifford theorem. Thus P (t) is modeled as follows: where Z is a normalizing constant, V c is a clique which is 469 defined as a subset of pixels such that any two distinct pixels 470 are mutual neighbors. Then Taking a negative logarithm of (6), the fusion problem can be 473 solved by minimizing the following function: The multi-479 algorithms fusion becomes minimizing the following energy 480 function: where N i contains top, bottom, left, right neighbor pixels 483 of i. The parameter α controls the preference towards sparser 484 tampering maps and the paraleter β controls the interaction 485 strength of neighboring pixels. 486 We use the graph-cut based solver [53], [54] from UGM 487 toolbox [55] to find the optimal tampering map. This 488 approach uses a MRF to model the prior thus we call it 489 interchangeably as MRF based fusion approach. Quantity Bel(A) can be interpreted as the degree to which 511 the evidence supports A. 512 In order to combine the evidence coming from multiple 513 independent sources of information, we can use Dempster's 514 combination rule to merge them. Let m 1 and m 2 be two mass 515 functions derived from independent items of evidence. They 516 can be fused to induce a new mass function m 12 defined as  where r n = y n −f (y n ) is the noise residual of the image y n , f is 555 a denoising filter. In the following, for the sake of simplicity, 556 we assume that the estimation of the camera PRNU has no 557 error, i.e.,k = k.

589
The SF based forgery detection is an approach in which 590 we first extract some inherent features of image blocks that 591 are likely to be modified when an image undergoes tam-592 pering and then use these features to proceed a two-class 593 pristine/forged training procedure. It can be said that this is 594 a universal approach in which we can detect many types of 595 forgeries though the accuracy is not high. Among various 596 statistical feature sets proposed in steganalysis, in this paper 597 we adopt the statistical features named Spatial Color Rich 598 Model (SCRM) [59] which work quite effectively in forgery 599 detection [33]. SCRM is an extension of SRM. The SRM 600 features from the R, G, and B channel are first added together 601 and then concatenated three dimensional co-occurrences of 602 residuals computed from all three color channels. These where K is the number of blocks containing pixel I i,j , and v k 617 is the vote score for the k th block that contains I i,j .

618
The framework proposed in this subsection aims at fusing 620 the evidence coming from the PRNU-based forgery detection 621 and the SF-based forgery detection. We believe that aggre-622 gating the evidence from the SF-based approach will help to 623 decrease the false alarm rate on the saturated and dark regions 624 of images. The fusion procedure can be described as follows.
The degree of conflict K and the fused mass function m 12 is 649 computed as follows The belief function in this case is equal to the fused mass  in eq. (8). We use the data term as in [26] and [27]:

668
In this subsection, we will report some preliminary experi-669 ments to compare the performance of two fusion techniques. 670 Our experiments were carried out on the dataset of the UTT 671 which includes of images taken from three cameras, a Canon 672 EOS-100D, a Nikon D5200 and a Panasonic DMC-GM1. The 673 first row of Fig. 1 is some examples of realistic tampering 674 images created by hand in modern photo-editing software 675 in which their original images taken from the dataset of 676 UTT. We first have estimated the PRNU of each camera 677 over 100 images and then have extracted 25000 correlation 678 samples over 25 images coming from other cameras and 679 25000 samples coming from the same camera to train the 680 correlation predictor as proposed in [58]. 681 We present in this section results only for one of the 682 cameras, a Canon EOS-100D. For our experiments we used 683 200 tampered images and 200 pristine ones. The forgeries 684 have been created with a copy-and-paste process and are 685 all rectangular with size of 128 × 128. We evaluated the 686 percentage of correctly detected forged pixels in the tampered 687 images (P D ) and the percentage of falsely identified pixels 688 in the pristine ones (P FA ), varying the relevant parameters of 689 the algorithms that is, the γ 1 , γ 2 in the PRNU-based forgery 690 detection algorithm, and the threshold λ in the DST fusion 691 and the parameter α, β in the Bayesian fusion.

692
In Fig. 1, we show the evaluation on several realistic tam-693 pered images (second row). The original images (first row) 694 are taken from Canon EOS-100D camera and then are forged 695 by inserting objects using the popular photo editing software 696 GIMP. The third and fourth columns show the output maps 697 of the SF and the PRNU based approaches. As can be seen, 698 each individual approach has its own limitation. The PRNU 699 based approach correctly detects the tampered regions but the 700 false alarm rate is hard to avoid due to the saturated regions 701 (see Fig. 2) and dark regions (see Fig. 3). In contrast, the SF 702 based approach does not localize tampered regions with high 703 accuracy but it does not have problem with saturated and dark 704 regions. The integrated map fused from the PRNU and SF 705 approaches in the fifth row shows a significant improvement 706 of the DST fusion method. The last row are the decision 707 maps integrated using MRF based fusion. We can see that, 708 on these selected forgery images, MRF based fusion performs 709 worse than DST based fusion did. It is worth mentioning that 710 we did not apply any morphological operation in the fusion 711 approach.

712
In Fig. 4 we show the ROCs (receiver operating char-713 acteristics) of the PRNU-based approach and of the fusion 714 VOLUME 10, 2022

728
It is noted that choosing what to fuse is also an art. It had 729 better to analyze the strength and weakness of each algorithm 730 before deciding to fuse them. As mentioned before, the weak-731 ness of PRNU algorithm is to make high false alarm rate on 732 the saturated and the dark regions while the SF based detec-733 tor does not. Therefore, integrating these two maps could 734 enhance the detection performance. If we test on the dataset 735 where there are not considerably saturated and dark regions, 736 it is hard to see the significant improvement because the SF 737 based algorithm does not have a chance of leveraging its 738 advantage. This explains why the tested results on 200 images 739 in general do not show much improvement comparing to the 740 results on 10 images whose saturated and dark regions are 741 considerable.    Beside that, we also use the F1-score to evaluate the The F1-score takes a high value when Precision and Recall 750 are both important. The higher F1-score, the more efficient 751 the algorithm is. Please see Table 2 for the meaning of the 752 measure used in forgery localization performance.  Table 3 shows the F1-score evaluated on the 10 forgery 754 images and 10 genuine images whose saturated and dark 755 regions are considerable. We can see that although the F1-756 score are low but it did show the significant improvement 757 of the fusion. The F1-score of DST fusion is highest and 758 10 times greater than those of PRNU and SF based detectors. 759 As mentioned in the introduction, the tampered parts of an 763 image could either be copied within the original image, the 764 so-called copy-move tampering, or come from another image, 765 the so-called copy-paste tampering. The copy-move detection 766 methods usually rely on duplicated regions detection thereby 767 failing to detect copy-paste tampering images. On the other 768 hand, algorithms aiming at detecting copy-paste tampering 769 are limited to detect copy-move tampering images. Motivated 770 from this idea, we come up with the idea integrating decision 771 maps generated from algorithm detecting copy-paste and 772 copy-move tampering to enhance performance. In the context 773 of the project DEFACTO, 2 we find it necessary to combine 774 the decision maps from different research teams [62], [63] to 775 enhance the detection performance. In [62] Le et al. proposed 776 to use demosaicing artifacts (also known as color filter array 777 (CFA) interpolation) to detect tampered parts of images. Par-778 ticularly, traces left by demosaicing are specific for different 779 camera brands and/or models. The lack of these traces or their 780 FIGURE 6. Fusion of forensic algorithms improves detection performance by detecting both cut-paste and copy-move forgery regions. These two tampering images are chosen from the dataset of Korus [26]. The first rows are original images, the second row are tampered images, the third row are ground-truth images, the fourth row are maps detected by [62], the fifth row are maps detected by [63] Fig. 6). Therefore it is a good idea 789 to fuse these decision maps to improve the performance.

802
The experiment is tested on images containing both 803 copy-move and copy-paste tampering chosen from the dataset 804 generated by Korus et al. [26]. Such chosen images would 805 leverage the performance of fusion algorithms because 806 the individual algorithms in [63] and in [62] fail to 807 detect copy-paste and copy-move tampering respectively (see 808 Fig. 6) while the integrated of these two algorithms could 809 detect both tampering operations.

810
The F1-score in Table 4 shows that the MRF based fusion 811 gives the highest F1-score while the DST based fusion gives 812 the lowest F1-score. Figure. 7 visually illustrates that the 813 MRF based fusion algorithm gives better performance than 814 the DST based fusion and individual algorithm [62], [63]. 815 This significant improvement of the MRF based fusion is 816 mainly based on the following reasons: First, the prior of the 817 tampering map is modeled with a MRF thereby exploiting 818 spatial dependencies of neighborhood pixels. This helps a 819 lot to decrease the number of non-detected tampered pixels 820 comparing to the algorithm [62]. For instance, comparing 821 to the ground-truth images in the third column of Fig. 7, 822 we see that the MRF fusion (the last column) is able to 823 detect splicing pixels that the algorithm in [62] missed (the 824 forth column). Second, the MRF fusion could integrate both 825 copy-paste and copy-move tampering parts (see first column 826 of Fig. 6 and first and last column of Fig. 7). Third, the DST 827 based fusion in this context fails to improve the performance 828 because the DST is limited to combine the conflict evidence. 829 More specifically, we are considering to combine the decision 830 maps of the algorithm [62] which could detect copy-paste 831 tampering but not copy-move and those of [63] which could 832 detect copy-move but not copy-paste tampering. Therefore, 833 there are usually the conflict parts in these maps. That is the 834 reason why the decision maps generated from DST fusion 835 (the sixth column of Fig. 7) are usually all black.

837
In this subsection, we will discuss the differences and sim-838 ilarities between the DST and MRF fusion techniques and 839 explain the advantages and disadvantages of each fusion 840 technique in two considered experiments.

841
Both fusion techniques have a certain initial requirement. 842 While the Bayesian technique requires the prior probabili-843 ties, the DST technique requires masses to be assigned in a 844 meaningful way to the various states, including an undecided 845 state. However, in this work we only consider two states 846 that are tampered and not tampered. The implementation of 847 DST fusion is quite simple comparing to the Bayesian fusion. 848 This paper has provided a systematic review on the state-of-888 the-art of fusion techniques applying in detecting and local-889 izing forgery images domains. We then have proposed two 890 effective fusion techniques, DST and Bayesian, to aggregate 891 the tampering maps. Two fusion scenarios have been consid-892 ered and experimental results have been tested on two differ-893 ent datasets. In the first scenario, the fusion method is applied 894 to aggregate the decision maps of PRNU based approach and 895 SF based approach. Preliminary experimental results have 896 shown that DST fusion method outperforms the Bayesian 897 fusion method on a particular dataset. This improvement is 898 mainly due to the fact that the DST fusion method has signif-899 icantly decreased the false positive rate on the saturated and 900 dark regions which is one of the most challenging limitation 901 of the PRNU based approach. In the second scenario, the 902 fusion method is applied to integrate the decision maps of the 903 algorithm based on demosaicing artifacts and the one based 904 on SIFT key-points and descriptors. The experimental results 905 have shown that MRF fusion has considerably performed 906 better than the DST fusion. The ability to exploit the spatial 907 dependencies of neighborhood pixels in the decision maps 908 has leveraged the detection performance of MRF based fusion 909 technique. 910 We have concluded that the final choice for a fusion 911 framework depends on the scenarios, the properties of each 912 individual forensic algorithm and requirements of the user.

913
In this paper, we have just considered the very basic setting 914 and conditions on two fusion methods. As a topic for further 915 research, we shall devote to analyzing more deeply on each 916 fusion method. Particularly, the limitation of the traditional 917 DST fusion when dealing with conflict evidence shall be stud-918 ied further [65]. Moreover, we shall consider more advanced 919 combination rules in DST fusion such as the transferable 920 belief model (TBM) [66] and Dezert-Smarandache theory 921 (DSmT) [67]. Various dataset and more prior information 922 for Bayesian method will be provided to have a thorough 923 comparison between these two methods. 924 VOLUME 10, 2022