A Review on Deep Learning Techniques for the Diagnosis of Novel Coronavirus (COVID-19)

Novel coronavirus (COVID-19) outbreak, has raised a calamitous situation all over the world and has become one of the most acute and severe ailments in the past hundred years. The prevalence rate of COVID-19 is rapidly rising every day throughout the globe. Although no vaccines for this pandemic have been discovered yet, deep learning techniques proved themselves to be a powerful tool in the arsenal used by clinicians for the automatic diagnosis of COVID-19. This paper aims to overview the recently developed systems based on deep learning techniques using different medical imaging modalities like Computer Tomography (CT) and X-ray. This review specifically discusses the systems developed for COVID-19 diagnosis using deep learning techniques and provides insights on well-known data sets used to train these networks. It also highlights the data partitioning techniques and various performance measures developed by researchers in this field. A taxonomy is drawn to categorize the recent works for proper insight. Finally, we conclude by addressing the challenges associated with the use of deep learning methods for COVID-19 detection and probable future trends in this research area. The aim of this paper is to facilitate experts (medical or otherwise) and technicians in understanding the ways deep learning techniques are used in this regard and how they can be potentially further utilized to combat the outbreak of COVID-19.


I. INTRODUCTION
OVEL coronavirus (COVID- 19), resulting from a severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has become a pandemic worldwide in recent times [1], [2].The number of infected cases as well as the death rate is increasing rapidly.It is reported that about 19,000,000 people have been infected with COVID-19, the death cases are around 700,000, and the number of recovered patients are around 10,000,000 globally [3].The universal transmission of COVID-19 has put a large amount of the world's population into quarantine, and ravaged numerous industrial sectors which in turn caused a worldwide financial crisis.
Reda Alhajj is with the Department of Computer Science, University of Calgary, Calgary, Alberta, Canada (e-mail: alhajj@ucalgary.ca).
in some scenarios, no symptoms are visible (asymptomatic) that make the disease an even bigger threat to public health.The reverse transcript polymerase chain reaction (RT-PCR) is considered as the gold standard for COVID-19 diagnosis [6].However, the lack of resources and strict test environment requirements restrict fast and effective screening of suspicious cases.Furthermore, RT-PCR inspection also experiences high false negative rates [7].Unfortunately, the only solution to effectively combat this transmissible disease, is through clinical vaccines as well as precise drug/therapy practices, which are not yet available.COVID-19 has proven to be amongst the most dangerous ailments that have posed severe threat to human civilization.With the evolution of modern technology in the past few decades, ingenious solutions have been created to assist disease diagnosis, prevention as well as control which leverage smart healthcare tools and facilities [8], [9], [10], [11].Specifically, for COVID-19 diagnosis, different imaging modalities like CT and X-ray are considered among the most effective techniques [12], [13], [14], when available, CT screening is preferred in comparison with X-rays because of its versatility and threedimensional pulmonary view [15], [16] though X-rays are must more affordable and widely available.These traditional medical imaging modalities play a vital role in the control of the pandemic.
Artificial Intelligence (AI), an evolving software technology in the area of medical image analysis has also directly helped combating the novel coronavirus [17], [18], [19] by efficiently providing high quality diagnosis results and dramatically reducing or eliminating man power.Very recently, deep learning and machine learning, two major areas of AI have become very popular in medical applications.Deep learning based support systems are developed for COVID-19 diagnosis using both CT and X-ray samples [20], [21], [22], [23].Some of the systems are developed based on pre-trained model with transfer learning [24], [25] and a few of them are introduced using customized networks [26], [27], [28].Machine learning [29], [30], and data science [31] are also the diverse areas that are actively used for corona diagnosis, prognosis, prediction, and the outbreak forecasting.Computer vision [32] has also contributed for the reduction of the severity of this pandemic.Moreover, Internet of things (IoT) [33], [34], big data [35], [36], and smartphone technology [37], [38] are extensively utilized to enable innovative solutions to fight against the spread of COVID-19.
The main aim of the paper is to review the recent developments of deep learning based COVID-19 diagnosis systems based upon data collected from medical imaging samples.A taxonomy is presented that classifies the reviewed systems based on pre-trained model with deep transfer learning and customized deep learning technique.We review the most vital schemes developed for the diagnosis of COVID-19 highlighting some aspects such as the data used for experiments, the data splitting technique, and the evaluation metrics.An open discussion with the challenges of existing deep learning based systems as well as a projection of future works is also presented.
The rest of the paper is as follows.Section II categorizes the reviewed systems for proper understanding.Section III explains the recent developed systems for COVID-19 diagnosis from both CT and X-ray samples using pre-trained model with deep transfer learning.Section IV demonstrates the custom network based COVID-19 diagnosis systems from both CT and X-ray.The discussion with challenges as well as possible future trends are depicted in section V. Lastly, the paper is concluded in section VI.

II. TAXONOMY OF DEEP LEARNING BASED COVID-19 DIAGNOSIS SYSTEMS
Deep learning techniques are able to explain complex problems by learning from simple depictions.The main features that have made the deep learning methods popular are the capability of learning the exact representations and the property of learning the data in a deep manner where multiple layers are utilized sequentially [39], [40].Deep learning methods are widely used in medical systems such as biomedicine [41], smart healthcare [42], drug discovery [43], medical image analysis [44], etc.
More recently, it is extensively used in the automated diagnosis of COVID-19 patients.In general, deep learning based systems are comprised of several steps such as data collection, data preparation, feature extraction and classification, and performance evaluation.The general pipeline of a COVID-19 diagnosis system based on deep learning is illustrated in Fig. 1.At the data collection stage, the patients from the hospital environment are considered as a participant.The data may have different forms but for COVID-19 diagnosis, imaging techniques like CT and X-ray samples are taken.The following necessary step is the data preparation that converts the data into an appropriate format.In this step, data pre-processing includes some operations like noise removal, resizing, augmentation, and so on.The data partitioning step splits the data into training, validation, and testing set for the experiment.Generally, cross-validation technique is utilized for data partitioning.The training data is used to develop a particular model that is evaluated by validation data, and the performance of the developed model is appraised by test data.The major step of deep learning based COVID-19 diagnosis is the feature extraction and classification.In this stage, the deep learning technique automatically extracts the feature performing several operations repeatedly, and finally, the classification is done based on class labels (healthy or COVID-19).Lastly, the developed system is assessed by some evaluation metrics like accuracy, sensitivity, specificity, precision, F1-score, and so on.
In this paper, a taxonomy of classifying COVID-19 diagnosis system is presented to facilitate the navigation of the landscape.Two different perspectives are applied which are related to the used deep learning techniques and the used imaging modalities (see Fig. 2).In this paper, we have reviewed a total of 45 COVID-19 diagnosis systems.Among them, 23 systems (51.11% of the total reviewed systems) used pre-trained model for diagnosis purposes and 22 (48.89% of the total reviewed systems) used custom deep learning techniques for COVID-19 diagnosis.From a different perspective, 25 reviewed systems used X-ray images (55.55% of the total reviewed systems) as data source, and the remaining 20 systems utilized CT scans (44.44% of the total reviewed systems).

III. PRE-TRAINED MODEL WITH DEEP TRANSFER LEARNING
A pre-trained model is one that has already been trained in fields similar to the context of the application.In transfer learning, weight and bias are transferred from a large trained model to a similar new model for testing or retraining.There are several advantages for using pre-trained models with deep transfer learning.In general, training a model from scratch for large dataset requires high computing power and is time consuming [45], [46].The pre-trained model with transfer learning enables the facility to speed up the convergence with network generalization [47], [48].Numerous pre-trained models that are utilized in transfer learning are designed for the large convolutional neural network (CNN).There are several pre-trained models which are used for COVID-19 diagnosis such as AlexNet [49], GoogleNet [50], SqueezeNet [51], different versions of Visual Geometry Group (VGG) [52], diverse kinds of ResNet [53], Xception [54], different forms of inception [55], diverse types of MobileNet [56], DenseNet [57], U-Net [58], etc.The systems developed for COVID-19 diagnosis are described next.

A. Diagnosis Using Computer Tomography (CT) Images 1) Diagnosis Based on Multiple Source Data
Wu et al. [59] introduced a deep learning based coronavirus screening framework using the concept of multi-view fusion.The system used a variant of CNN called ResNet50.The dataset is collected from two different hospitals in China.A total of 495 images are taken into account for the experiment in which 368 are associated with confirmed COVID-19 cases, and 127 are of other pneumonia.In this scheme, the dataset is divided into a proportion of 80%, 10%, and 10% for training, testing, and validation respectively.Each of the images, considered in the system are resized into 256×256 before the network development.In the test case, the developed system obtained

B. Diagnosis Using X-ray Images 1) Diagnosis Based on Multiple Source Data
Apostolopoulos and Bessiana [69] developed a system for the automatic diagnosis of COVID-19 cases utilizing the concept of transfer learning with five variants of CNNs.The pre-trained models which are used in the system are VGG19, MobileNetv2, Inception, Xception, and Inception-ResNetv2.The system considered 1427 images including 224 for COVID-19, 700 for common pneumonia, and 504 for healthy cases in the first scenario.In the second scenario, 224 COVID-19 images, 714 bacterial and viral pneumonia images, and 504 healthy individual images are considered.The dataset was divided using the 10-fold cross-validation method.It was revealed that the highest accuracy of 96.78%, sensitivity of 98.66%, and specificity of 96.46% are obtained for the second dataset using MobileNetv2.In another research work, Loey et al. [70] introduced a novel system for the diagnosis of coronavirus using Generative Adversarial Network (GAN) and pre-trained models of CNN with deep transfer learning.The pre-trained models which are used in the proposed system are Alexnet, Googlenet, and Resnet18.As the number of X-ray images for COVID-19 is small, GAN is used to generate more samples for accurate detection of this virus.A total number of 307 images are considered including four classes like COVID-19, normal, pneumonia_bac, and pneumonia_vir.The system experimented on three different scenarios of dataset depending on the consideration of class level.Considering four classes, Googlenet obtained the highest accuracy of 80.6%.Alexnet and Googlenet achieved accuracy of 85.2% and 100% respectively considering three and two classes.Horry et al. [75] described a COVID-19 detection framework using the concept of pre-trained model in X-ray images.The proposed system used four popular pre-trained models like VGG, Inception, Xception, and Resnet with transfer learning.The used dataset consisted of 100 COVID-19 cases, 100 pneumonia, and 200 healthy cases for experiments.In this system, a ratio of 80:20 is preserved for training and testing set as a data partition.The experimental findings reveal that the system obtained precision, sensitivity, and F1-score of 83%, 80%, and 80% respectively using VGG-19 which is measured as the highest performance in the study considering three-class data.Further, Ozcan [76] proposed a deep learning scheme with a combination of the grid search strategy and three pre-trained models of CNN.The used pre-trained models are GoogleNet, ResNet18, and ResNet50.The grid search technique is used to select the best hyperparameter and the pre-trained models are utilized for feature extraction and classification.The system used three public datasets where the images are of 242 bacteria cases, 131 COVID-19 cases, 200 normal cases, and 148 viral cases.All the data are partitioned into training, testing, and validation set in a proportion of 50:30:20.The ResNet50 with grid search performed better and obtained accuracy of 97.69%, sensitivity of 97.26%, specificity of 97.90%, precision of 95.95%, and F1-score of 96.60%.
Sethy and Behra [77] introduced a system for the diagnosis of COVID-19 cases using pre-trained models of CNN and Support Vector Machine (SVM).The algorithm used eleven CNN pre-trained models for automatic extraction of features, and SVM for classification.In this system, two separate datasets were used where the first dataset included 25 positive COVID-19 and 25 negative X-ray images of COVID-19.A total of 133 images containing Middle East Respiratory Syndrome (MERS), SARS, and Acute Respiratory Distress Syndrome (ARDS) are used as positive samples and 133 normal X-ray images as negative samples in the second dataset.From the experimental results, it is found that Resnet50 with SVM obtained accuracy, False Positive Rate (FPR), Matthews Correlation Coefficient (MCC), and Kappa of 95.38%, 95.52%, 91.41%, and 90.76% respectively which the best is in the developed system for the first scenario of the dataset.Minaee et al. [78] proposed a framework named Deep-COVID using the concept of deep transfer learning for COVID-19 prediction in X-ray images.Four popular pre-trained models like ResNet18, ResNet50, SqueezeNet, and DenseNet-121 were considered in this study for COVID-19 diagnosis.In total, 5071 images are collected from different open-access resources.Among them, 2000 images with 31 COVID-19 cases were used for training, and 3000 images with 40 COVID-19 infected cases were used for testing in the experiments.The resulting dataset was named COVID-Xray-5k.The best performance obtained by the system is sensitivity of 100%, and specificity of 95.6% using SqueezeNet.
In another study, Punn and Agarwal [79] developed an automated COVID-19 diagnosis system using several pretrained models like ResNet, Inception-v3, Inception ResNet-v2, DenseNet169, and NASNetLarge with a small number of X-ray images.The system used random oversampling and weighted class loss function for fine-tuning called transfer learning.In this system, a total of 1076 chest X-ray images are considered for experiments.The dataset is partitioned into 80%, 10%, and 10% ratio for training, testing, and validation set respectively.From the experimental results, it was shown that NASNetLarge performed comparatively better and achieved accuracy, precision, sensitivity, AUC, specificity, and F1-score of 98%, 88%, 91%, 99%, 98%, and 89% respectively.Afterward, Narin et al. [80] introduced a method for automatically classifying COVID-19 infected patients from X-ray images using the variants of CNN.The pre-trained models used are ResNet50, InceptionV3, and Inception-ResNetV2 which obtained higher predictive accuracy on a subset X-ray dataset.The system used a total of 100 X-ray images where 50 images were from COVID-19 patients while the remaining 50 from healthy individuals.The 5-fold cross-validation was used to partition the dataset for the experiment.The system achieved an accuracy of 98%, 97%, and 87% from ResNet50, InceptionV3, and Inception-ResNetV2 respectively in test cases.In terms of other evaluation metrics, the best performance was obtained using RecNet50 with a recall of 96%, specificity of 100%, precision of 100%, and F1-score of 98%.
Bukharia et al. [81] presented a COVID-19 diagnosis system using a variant of CNN named Resnet50.The system considered 278 X-ray images of three classes where 89   In another research project, Moutounet-Cartan [94] developed a deep learning based system to diagnose the novel coronavirus as well as other pneumonia diseases from X-ray images.The system used the following variants of CNN architecture named VGG-16, VGG-19, InceptionResNetV2, InceptionV3, and Xception for diagnosis.In this study, in total 327 X-ray images were taken where 152 cases were from healthy people, 125 from COVID-19 cases, and the remaining 50 cases from other pneumonia diseases.The dataset is partitioned using the principle of 5-fold cross-validation.The system found VGG-16 as the best performing model and obtained overall accuracy of 84.1%, sensitivity 87.7%, and AUC of 97.4% where the sensitivity and AUC were considered only for COVID-19 cases.Furthermore, Maguolo and Nanni [95] evaluated the performance of COVID-19 detection system from X-ray samples utilizing a popular pertained model named AlexNet.The system used four different publicly available datasets to evaluate the performance.A total of 339,271 images were taken where 144 images for COVID-19 patients, 108,948 samples of pneumonia and bacteria except COVID-19, 224,316 chest radiographs of bacteria and pneumonia, and 5,863 paediatric images viral and bacterial pneumonia.The dataset was partitioned into 10-fold cross-validation for training and testing.Using the concept of deep transfer learning, the system obtained the highest AUC of 99.97% in the study.

2) Diagnosis Based on Single Source Data
Very recently, Hemdan et al. [97] proposed a system named COVIDX-Net to diagnose coronavirus using the variants of CNN in X-ray images.A total of seven pre-trained models are considered in this study.The dataset consisted of 50 images where 25 images are from healthy people and the remaining 25 samples from COVID-19 cases.For the experiment, the dataset was split into a proportion of 80% and 20% for training and testing set respectively.The experimental results revealed that VGG-19 and DenseNet outperformed the other pre-trained models with an accuracy of 90% and F1-score of 91%.InceptionV3 obtained the worst results.
Table II summarizes the aforementioned deep learning based COVID-19 diagnosis systems from X-ray samples using pretrained model with deep transfer learning and describes some of the significant factors, such as data sources, number of images and classes, data partitioning technique, the used techniques for diagnosis, and the performance measures of the reported systems.

IV. CUSTOM DEEP LEARNING TECHNIQUES
Custom deep learning techniques provide the ability to develop a user-friendly architecture and to allow for more consistent and accurate performance due to the attention to the specific application of interest.The custom networks are evolved with the use of a particular deep learning method [98] or the hybridization of deep learning algorithms [99], [100] or the hybridization of deep learning with other fields of AI such as machine learning, data mining, nature-inspired algorithms, etc. [101], [102].No previous weights and bias are used in the customized network like pre-trained model hence it requires comparatively high computation power and time.The systems developed for COVID-19 diagnosis are outlined as follows.

A. Diagnosis Using Computer Tomography (CT) Images 1) Diagnosis Based on Multiple Source Data
Elghamrawy and Hassanien [103] proposed a scheme for the diagnosis and prediction of coronavirus infected patients using a combination of CNN and Whale Optimization Algorithm (WOA) from CT samples.In the proposed method, CNN is used for diagnosis, and WOA is utilized for prediction.The used dataset is collected from publicly available databases consisted of 617 CT scans.Among them, 134 images are excluded as it contains non-lung region.A total of 432 images confirmed of COVID-19 and 151 cases of other viral pneumonia were considered.To achieve better performance, the dataset was divided into a proportion of 65%, and 35% for training and testing respectively.The proposed system obtained overall accuracy, sensitivity, and precision of 96.40%, 97.25%, and 97.3% respectively for diagnosis.Further, He et al. [104] proposed a deep learning method named CRNet for the detection of COVID-19 using CT images.In this system, a total of 746 CT images were analysed where 349 were associated with COVID-19 cases, and 397 with non-COVID-19 cases.The dataset was formed by merging three publicly available datasets which was divided into three sets named training, testing, and validation set in a proportion of 60%, 25%, and 15% respectively.The proposed system obtained accuracy of 86%, F1-score of 85%, and AUC of 94% from the experimental results.In comparison with other prominent pre-trained models, the proposed system used comparatively less tuning parameters.
Afterwards, Wang et al. [105] introduced a scheme for COVID-19 diagnosis using a modified CNN technique named modified-Inception.The basic difference between Inception and modified-Inception is that modified-Inception reduces the dimension of attributes before final classification.In the experiment, the scheme used 1040 CT images, in which 740 were tagged as COVID-19 positive and 325 as COVID-19 negative.The dataset was partitioned into training, testing, and validation set randomly.The experimental outcomes revealed that the scheme achieved accuracy, sensitivity, specificity, precision, and F1-score of 79.3%, 83%, 67%, 55%, and 63% respectively on the testing samples.Moreover, Liu et al. [106] developed an automatic COVID-19 diagnosis system using deep learning method via CT images.The system used modified DenseNet-264 (COVIDNet) for diagnosis where the model consisted of 4 dense blocks.In this system, 920 COVID-19 and 1,073 non-COVID-19 cases were considered for the experiment.To obtain better performance, the dataset is partitioned into three sets namely training, testing, and validation in a proportion of 60%, 20%, and 20% respectively.The developed system obtained accuracy of 94.3%, AUC of 98.6%, sensitivity of 93.1%, specificity of 95.1%, precision of 93.9%, NPV of 94.5%, and F1-score of 93.5%.
In another study , Ying et al. [107] introduced a deep learning technique based on the Details Relation Extraction neural network (DRE-Net) named Deep Pneumonia for the diagnosis of COVID-19 cases utilizing CT images.The dataset was collected from two popular hospitals in China.In this system, a total of 1990 image slices were taken where 777 images for COVID-19, 505 slices for bacterial pneumonia, and 708 samples from normal people.The dataset was split in a proportion of 60%, 30%, and 10% for training, testing, and validation set respectively.The proposed system obtained accuracy of 94%, sensitivity of 93%, precision of 96%, F1score of 94%, and AUC of 99%.To detect COVID-19, Zheng et al. [108] proposed a 3D deep convolution neural network (DeCoVNet) from CT scans.The proposed network is comprised of three segments like a vanilla 3D convolution, a batch norm layer, and a subsampling layer.The data for the study was collected from the hospital environment.A total of 630 CT samples were used for experiment where 80% (499 images) were in the training set, and the rest 20% (131images) were used in testing.From the experimental outcome, accuracy, sensitivity, specificity, precision, NPV, AUC of 90.1%, 90.7%, 91.1%, 84%, 98.2%, and 95.9% are achieved.
Hasan et al. [109] proposed a hybrid system using the concept of Q-deformed entropy and deep learning features (QDE-DF) to differentiate COVID-19 infected people from pneumonia cases, and healthy people utilizing CT images.For deep features extraction, CNN and Q-deformed entropy were used, and LSTM was used to classify the cases from deep features.A total of 321 chest CT samples were used for this study, consisting of 118 CT samples of COVID-19 cases, 96 CT samples of pneumonia cases, and 107 CT samples of healthy individuals.Approximately, 16 attributes were extracted from each image using a feature extraction technique.To assess the developed system, the dataset was partitioned in a proportion of 70%, and 30% for training and testing set respectively.The system obtained accuracy of 99.68% which is considered as the highest in this study.Further, Amyar et al. [110] developed a scheme using deep learning method to diagnose COVID-19 patients from CT samples.The system consists of an encoder for reconstruction and two decoders for segmentation, and for classification purposes, a multi-layer perceptron is used.The used dataset included 1044 cases where 449 cases were of confirmed COVID-19, 100 cases from healthy individuals, 98 samples were from confirmed lung cancer patients, and 397 from various other kinds of pathology.Collectively, 449 were associated with COVID-19 and 595 were not.The dataset was partitioned into training, validation, and testing set in a ratio of 80%, 10%, and 10% respectively.The proposed system received accuracy of 86%, sensitivity of 94%, specificity of 79%, AUC of 93%.

B. Diagnosis Using X-ray Images 1) Diagnosis Based on Multiple Source Data
Ozturk et al. [121] presented a customized network (DarkCovidNet) for the automatic diagnosis of COVID-19 in raw chest X-ray samples utilizing deep neural networks.The proposed system used DarkNet as a classifier with 17 convolutional layers.In this system, two sources of the dataset were used which includes 127 images from first sources, and 500 normal and 500 pneumonia cases from frontal X-ray samples from the second source.The dataset was partitioned in 5-fold cross-validation technique.The obtained sensitivity, specificity, precision, F1-score, and accuracy of 95.13%, 95.3%, 98.03%, 96.51%, and 98.08% respectively for binaryclass which are the highest in this study.Moreover, Ucar and Korkmaz [122] developed a COVIDiagnosis-Net based on the Bayes-SqueezeNet for the diagnosis of coronavirus utilizing Xray samples.The system used 1591 pneumonia cases with non-COVID-19, 45 COVID-19 cases, and 1203 uninfected normal patients in total as the dataset.The dataset is formed with the combination of three publicly available datasets.From the total data, 80% for training, 10% for validation, and 10% for testing are used in the proposed system.The experimental results obtained accuracy, correctness, completeness, specificity, f1score, and MCC of 98.26%, 98.26%, 98.26%, 99.13%, 98.25%, and 97.39% individually in overall.
In another study, Wong and Wang [123] developed a coronavirus detection mechanism from chest x-ray data called COVID-Net.The system generated a dataset COVIDx by combining and modifying two open-access datasets.In this study, the dataset consisted of a total of 13, 800 chest X-ray samples from 13,645 patients.The system considered three classes by combining bacterial and viral classes into a negative case.Among the total data, 90% was used for training and the rest 10% was utilized for validation.The proposed network obtained 92.4% accuracy in 10 iterations for test cases, and the sensitivity and precision of 80% and 88.9% were achieved in the case of COVID-19 class.Further, Khan et al. [124] proposed a deep CNN architecture named CoroNet for the diagnosis of COVID-19 infected patients from chest X-ray radiographs.In this system, a total of 1300 images were considered where 290 samples of COVID-19, 660 of bacterial pneumonia, 931 of viral pneumonia, and 1203 of normal patients.The dataset was split at a proportion of 80% and 20% for training and validation set respectively.The proposed system obtained accuracy, precision, sensitivity, and F1-score of 89.5%, 97%, 100%, and 98% respectively for COVID-19 class.
Recently, Rahimzadeh and Attar [125] proposed a modified CNN network for the diagnosis of novel coronavirus cases using X-ray samples.The system concatenated two well-known architecture of CNN named Xception and ResNet50V2 that make the system robust using multiple features extraction capability.Among the 15085 images, 180 were confirmed COVID-19, 6054 were pneumonia, and 8851 were normal cases.The scheme used 5-fold cross-validation for data partitioning.The network obtained accuracy of 99.50%, sensitivity of 80.53%, specificity of 99.56%, and precision of 35.27% for COVID-19 detection.Furthermore, Mukherjee et al. [126] proposed a system for the detection of novel coronavirus using shallow CNN in chest X-ray radiographs.The developed network is comparatively light-weight due to a small number of parameters.In this system, 130 positive COVID-19 cases, and 130 non-COVID cases were considered where the non-COVID cases include MERS, SARS, pneumonia, and normal chest X-rays.To obtain better performance, the dataset was split using 5-fold cross-validation.The performance of the system was evaluated by tuning the batch size of the CNN architecture.From the experimental results, it is found that the system obtained the highest accuracy, sensitivity, specificity, precision, F1-score, and AUC of 96.92%, 94.20%, 100%, 100%, 97.01%, and 99.22% respectively for batch size 50.
In another study , Li et al. [127] introduced a robust technique for automatic COVID-19 screening using discriminative costsensitive learning (DCSL).DCSL is formed with the combination of fine-grained classification and cost-sensitive learning.The used dataset consisted of 2,239 chest X-ray samples where 239 samples of COVID-19 cases, 1,000 samples from bacterial or viral pneumonia cases, and 1,000 samples of normal people.To obtain better performance, the dataset was partitioned using 5-fold cross-validation method.The proposed system achieved accuracy of 97.01%, precision of 97%, sensitivity of 97.09%, and F1-score of 96.98%.Khobahi et al. [128] developed a semi-supervised deep learning system based on Auto-Encoders named CoroNet to detect COVID-19 infected patients.The proposed system merged three openaccess datasets for experiments.In this scheme, 18,529 images of different categories were used.Among the images, 99 samples were of COVID-19 classes, 9579 were of non-COVID pneumonia, and 8851 samples were related to healthy cases.The dataset was split in a proportion of 90% and 10% for training and testing set respectively.Overall, the accuracy, precision, recall, and F1-score of 93.50%, 93.63%, 93.50%, and 93.51% were achieved from the experiment.Moreover, Luz et al. [130] presented an efficient deep learning scheme named EfficientNet for the detection of coronavirus pattern from X-ray radiographs.The main advantage of EfficientNet is that it used fewer parameters, approximately 30 times less parameters than the pre-trained model.The system considered 30,663 images for experiment where 183 cases considered as COVID-19,

2) Diagnosis Based on Single Source Data
Alqudah et al. [131] suggested a hybrid method for the diagnosis of patients affected with coronavirus from X-ray data.The proposed system combined deep learning (CNN) and machine learning (SVM, RF) architecture.Deep learning was utilized both for feature extraction and classification task where machine learning was only used for classification task.In the system, 71 X-ray images of the chest were used in total where 48 are used for positive and 23 for negative.The system used 70% for training and 30% for testing and experiments were performed on several combinations like CNN-Softmax, CNN-SVM, and CNN-RF.The system obtained accuracy, sensitivity, specificity, and precision of 95.2%, 93.3%, 100%, and 100% respectively for CNN-Softmax classifier.Afterward, Farooq and Hafeez [132] presented a deep learning scheme with a pre-trained ResNet-50 network to detect the COVID-19 infected patients named COVID-ResNet.The proposed system implemented a residual neural network with 50 layers in total.In this study, the dataset comprised of 13,800 chest X-ray samples from 13,645 patients in total.The system achieved accuracy of 96.23% overall with 41 iterations using the COVIDx dataset.The other evaluation metrics like sensitivity, precision, and F1-score of 100%, 100%, and 100% are obtained considering the COVID-19 case only.Further, Afshar et al. [133] developed a capsule network based system named COVID-CAPS for the diagnosis of COVID-19 patients using X-ray samples.The proposed framework is well suited to work with a small dataset.The dataset used in the network was the COVIDx dataset, which is popular for COVID-19 research.The developed scheme used 13,800 chest x-ray images from 13,645 patients for experiments.Although there are four classes in the dataset, the system classifies the images into three classes considering bacterial and viral into one negative class.COVID-CAPS obtained accuracy, sensitivity, specificity, and AUC of 95.7%, 90%, 95.8%, and 0.97 respectively.
Table IV summarizes the aforementioned deep learning based COVID-19 diagnosis systems from X-ray samples using custom deep learning techniques and demonstrates some of the important factors, such as data sources, number of images and classes, data partitioning technique, diagnosis techniques, and the evaluation metrics of the developed systems.

V. OPEN DISCUSSIONS, CHALLENGES AND FUTURE TRENDS
This section demonstrates the discussions of reviewed systems, challenges, and the possible future trends of deep learning based systems for COVID-19 diagnosis.

A. Open Discussions
In this paper, 45 systems were reviewed where 23 systems used pre-trained model as a deep learning architecture, and the remaining 22 utilized custom deep learning framework.The results of the individual system are presented for explanation.Two popular imaging techniques CT and X-ray are used for data samples.Among the reviewed system, 25 systems are developed based on X-ray data and the rest 20 used CT samples.The majority of the systems used multiple source data and a few of them used single source data.We summarized the developed systems considering some features like the data sources, the number of images and classes, the data partitioning techniques, the used deep learning technique for diagnosis, and finally the evaluation metrics for performance measure.The data sources are the benchmark dataset or real-time data from hospital environment.Some of the systems used a huge number of images but the number of samples for COVID-19 cases is comparatively small.Both the binary and multi-class are considered throughout the review.With respect to data partitioning, some of the systems used cross-validation techniques, and others used hold-out method.Both the pretrained model and custom deep learning architecture are taken into consideration.Almost all the systems used CNN or variants of CNN for diagnosis.Some common evaluation metrics like accuracy, sensitivity, specificity, precision, F1-score, AUC, etc. are utilized through the whole review.
The summary for the CT scan based COVID-19 diagnosis utilizing pre-trained model as well as customized deep learning technique is illustrated in Table I and Table III.It is evident from the results that most of the developed systems used realtime data from hospital environment of China and a few of the systems [61], [64], [103], [104], [109], [110], [111], and [119] used benchmark data.A few of the developed systems [66], [67], [68], [111], and [119] used data from a single source and the majority of the developed schemes used multiple source data.The datasets which are used multiple times in the reviewed systems are COVID-CT [112], and COVID-19 [113].The reviewed systems which used maximum and minimum number of images for the experiment are [64], and [119] where the COVID-19 cases are of 32,230 (augmented data), and 51 respectively.In case of the number of classes to be classified, most of the developed systems considered binary class (COVID-19, and non-COVID-19) while some of them [60], [63], [64], [107], and [109] considered multiple classes (COVID-19, pneumonia, and normal).The 10-fold [119] was taken into consideration in some cases, whereas some developed systems [61], [62], [67], and [105] used random partitioning, and the majority of them considered hold-out method for data splitting.As far as performance is concerned, the system developed in [66] obtained 100% sensitivity.Among the reviewed systems, most of the frameworks [61], [62], [63], [65], [66], [103], [106], [107], [109] achieved comparatively higher accuracy, sensitivity, specificity, precision, F1-score and AUC having these measure (where applicable) greater than 90%.The highest accuracy of 99.51% and 99.68% were found at [66], [119] using pre-trained model and customized network respectively.
Table II and Table IV depict the X-ray based diagnosis of COVID-19 using pre-trained model with deep transfer learning and customized deep learning architecture.Our analysis revealed that almost all the developed systems used a common dataset COVID-19 X-ray image database [82].Some systems used Kermany et al. [84] dataset, and COVIDx Dataset [123] frequently for diagnosis.All the proposed schemes utilized benchmark data for the experiment, no system applied real-time data.The systems introduced in [97], [131], [132], and [133] used single source data whereas the rest of the reviewed systems considered data from multiple sources.The framework demonstrated in [95] considered the highest number of images where the lowest case was used in [97].The number of COVID-19 cases for [95], and [97] is 144, and 25 respectively.Although some systems considered the maximum number of images but the total number of images for COVID-19 is comparatively small.In terms of number of class consideration, some of the systems [70], [76], [124], and [131] considered 4 classes, a few of them [69], [75], [79], [81], [91], [94], etc. used 3 classes, and the remaining reviewed systems utilized binary class for experiment.The cross-validation technique such as 10-fold [69], [95], and 5-fold [80], [94], [121], [125], [126], [127] are used in some cases, and other systems considered hold-out method for data partitioning.In case of performance measure of the developed systems, 100% accuracy was achieved by the proposed systems in [70], 100% sensitivity in [70], [78], [97], [124], and [132], 100% specificity in [80], [97], [126], and [131], 100% precision in [70], [80], [97], [126], [130], [131], and [132], 100% F1-score [70], and [132], 100% AUC obtained in [80].Most of the developed systems performed better in case of precision measurement, and the second best is obtained in case of sensitivity.Though the reviewed systems achieved comparatively better results for X-ray case both for pre-trained and custom network, the developed systems are not real-time tested with target people.
In comparing the pre-trained model with the custom network, some of the reviewed systems performed better for custom network.The performance of the developed systems varied depending on the dataset.The performance of the reviewed systems is not comparable as almost all the systems used different data sizes for the experiment.In terms of imaging modalities comparison, X-ray based systems performed comparatively better than CT based systems.But most of the X-ray based frameworks used benchmark data while the realtime data from hospital environment is considered in case of CT based systems.It is envisaged that the systems introduced using CT samples are applicable for real-time testing but the X-ray based proposed schemes need real-time testing with target people before application

B. Challenges and Future Trends
There are many unique challenges for applying deep learning techniques and algorithms for the detection of novel coronavirus (COVID-19).
While deep learning techniques are highly automatable, it needs a large set of data to develop a robust system for diagnosis purpose.As the COVID-19 is very new to research, the lack of standard data is a major challenge for diagnosis.On the other hand, the available imaging data for COVID-19 patients are incomplete, noisy, ambiguous, and inaccurate labels in some cases.To train a deep learning architecture with such massive and diverse data sets is very complex and there is a need to resolve a variety of problems like data redundancy, sparsity as well as missing values.Almost all the reviewed systems used different data sets for the experiment.The developed systems collected data from internet sources, prepared the data in their way, and finally evaluated their systems using evaluation metrics.For this reason, it is quite difficult to conclude definitively which system yields the best result for COVID-19 detection.
The further challenges for the COVID-19 detection systems are the imbalance in the dataset samples.It is a critical issue as there are a few COVID-19 samples both CT and X-ray whereas pneumonia and normal cases contain a huge number of samples than COVID-19 cases.It is evident from the reviewed systems proposed in [61], [62], [78], [95], etc.The imbalance in data very often raises bias during the training phase of deep learning technique.With the fewer number of positive samples, it has become increasingly difficult to balance out the target sample.The lack of confidence interval is another challenge found in deep learning based COVID-19 diagnosis systems.Deep learning architecture provides the output as prediction confidence whereas the output indicator of a particular neuron is considered as a single probability.For COVID-19 diagnosis, the lack of confidence interval across a predicted value is usually not desirable.
To overcome these challenges, researches may consider the design of optimized deep learning algorithms that can easily cope up with a small number of data [26].A shallow long shortterm memory (LSTM) [134] is utilized to solve the limitations of a small dataset.In the absence of large-scale training datasets, leveraging the current deep learning architecture as feature extractors and then conducting more learning on those attributes using end to end manner [135], [136] is a more encouraging path.Freezing is a technique that provides the facilities to shrink the number of parameters in deep learning architecture where the reduced parameters are hired from another network trained for similar purposes.While the number of parameters would reduce, it might be possible to achieve good performance from a small number of COVID-19 cases [137], [138].Ensemble learning [139], [140], and multi-task learning [110], [141] are better suited for COVID-19 diagnosis in the context of a small number of data.In case of ensemble learning, multiple architectures are developed instead of a single network and finally, the results of each network are combined.In multi-task architecture, diverse tasks are combined to take the facility of data annotations from one another.
Additionally, synthetic data generation might be a possible solution to overcome the challenges of deep learning based COVID-19 detection systems.The most used techniques for data generation are data augmentation and GANs which are frequently utilized to solve the class imbalance problem.The data augmentation technique [142], [143] generate new lesions from the given COVID-19 samples using flipping, rotation, cropping, random noise addition, etc. from the given images.But the overfitting problem may arise in case of augmented data.GANs are the most sophisticated techniques for realistic synthetic data generation.From a small number of COVID-19 samples, GANs [70], [144] generate a large number of images that are used to train a deep learning system for the novel virus diagnosis.Further, weakly supervised deep learning methods [145], [146] would be a probable solution for limited training data.Furthermore, as the manually labeling of COVID-19 imaging data is costly, and lengthy process, the use of selfsupervised deep learning techniques [147], [148] are highly recommended.

VI. CONCLUSION
The COVID-19 is still an ongoing pandemic that is creating new records in terms of cumulative and daily numbers for global infection and death.Deep learning based automatic diagnosis of COVID-19 which provides consistent and accurate solutions, has played a significant role to assist with the diagnosis of this disease.This paper presents the recent works for COVID-19 diagnosis purposes using deep learning techniques from two types of imaging techniques like CT and X-ray samples.The review describes the systems which are developed based on pretrained model with deep transfer learning and customized deep learning architecture for COVID-19 diagnosis.Two-leveled taxonomy was presented which explores the perspectives of deep learning techniques and imaging modalities.This paper outlines all the sources of used datasets which can be easily understood and accessed by the research community.The major challenge of the COVID-19 diagnosis systems based on deep learning is the lack of gold standard.Furthermore, the possible solutions to overcome the current challenges are recommended that might encourage the researchers who would like to contribute in this area.It is prudent to note that the deep learning techniques with imaging modalities offer only partial details about the infected patients.However, it is not really envisaged in the present state that the role of physicians or clinicians in clinical diagnosis can be replaced by deep learning techniques.In the near future, it is hoped that deep learning experts with radiologists would provide appropriate support systems for identifying the COVID-19 infected patients.

Fig. 2 .
Fig. 2. Taxonomy of the recent developed COVID-19 diagnosis systems using deep learning.accuracy of 76%, sensitivity of 81.1%, specificity of 61.5%, and Area under Curve (AUC) of 81.9%.The results are analysed both for single-view and multi-view fusion model but the multi-view fusion model demonstrates a superior performance.In another research, Xu et al. [60] developed a system for classifying healthy individuals from COVID-19 pneumonia and Influenza-A viral pneumonia utilizing CNN variants.The used pre-trained model in this system is Resnet18.The data is collected from three different hospitals in China.This study considers 618 CT images in which 219 images are obtained from patients infected with COVID-19, 224 from Influenza-A viral pneumonia, and 175 from normal individuals.To train the model, a total of 85.4% (528) images are used, and the remaining samples are used to test the developed model.The framework achieved 86.7% accuracy, 81.5% sensitivity, 80.8% precision, and 81.1% F1-score from the experiment.Afterward, Jin et al. [61] developed an artificial intelligence based coronavirus diagnosis system using a variant of CNN named ResNet152.The pre-trained model used 152 convolutional, subsampling, and fully-connected layers.The used dataset is collected from three renowned hospitals of China, and two publicly available databases.A total number of 1881 of cases are considered where 496 cases are for COVID-19 infected patients, and 1385 are negative cases.The dataset is split randomly for experiments.The system achieved an accuracy of 94.98%, sensitivity of 94.06%, specificity of 95.47%, precision of 91.53%, F1-score of 92.78, and AUC of 97.91% from the experiment.Furthermore, Jin et al. [62] introduced a medical system for COVID-19 screening using deep learning techniques.Their system used various pre-trained models of CNN like DPN-92, Inception-v3, ResNet-50, and Attention ResNet-50 with 3D U-Net++.The dataset is retrieved from different five hospitals in China.In this system, a total of 139 samples are used where 850 samples from COVID-19, and but Md.Milon Islam and Fakhri Karray are with the Centre for Pattern Analysis and Machine Intelligence, Department of Electrical and Computer Engineering,

TABLE I SUMMARY
OF DEEP LEARNING BASED COVID-19 DIAGNOSIS IN CT IMAGES USING PRE-TRAINED MODEL WITH DEEP TRANSFER LEARNING samples of COVID-19 infected, 93 samples of healthy participants, and 96 samples of pneumonia patients.The collected dataset was split into two sets like training and testing in a proportion of 80% (223 images), and 20% (55 images).The diagnosis process obtained accuracy, precision, recall, and F1-score of 98.18 %, 98.14%, 98.24%, and 98.19 % respectively from the experiment.

TABLE II SUMMARY
[91]EEP LEARNING BASED COVID-19 DIAGNOSIS IN X-RAY IMAGES USING PRE-TRAINED MODEL WITH DEEP TRANSFER LEARNING Abbas et al.[91]categorized COVID-19 infected patients, from healthy individuals using Decompose, Transfer, and Compose (DeTraC) deep ResNet18.The proposed DeTraC can fix any anomalies in the image dataset by using a class decomposition method to investigate its class boundaries.In this system, a total of 196 images were utilized where 80 samples of normal patients, 105 samples of COVID-19, and 11 samples of SARS.The system generated 1764 samples from given samples using decomposition.The dataset was split into two groups, 70% for system training and 30% for evaluation.

TABLE III SUMMARY
OF DEEP LEARNING BASED COVID-19 DIAGNOSIS IN CT IMAGES USING CUSTOMIZED NETWORK with developed hybrid composite extraction method.The system considered two classes of data named COVID-19, and SARS, each of the class comprised of 51 images.The dataset was partitioned using 10-fold crossvalidation technique to obtain a better outcome.The developed system obtained accuracy, precision, f1-score, and AUC of 94.11%, 99.4%, 94%, and 99.4% respectively.TableIIIsummarizes the aforementioned deep learning based COVID-19 diagnosis systems from CT samples using custom deep learning techniques and demonstrates some of the important factors, such as data sources, number of images and classes, data partitioning technique, diagnosis techniques, and the evaluation metrics of the developed systems.
[119]um cases.The system obtained accuracy of 93.25%, sensitivity of 90.70%, specificity of 90.72%, F1-score of 89.96%, and Kappa of 90.60% from the experiment.In another work, Farid et al.[119]introduced a new approach for classifying COVID-19 infection using the attributes from CT images.The image parameters were taken using four image filters in mixture

TABLE IV SUMMARY
OF DEEP LEARNING BASED COVID-19 DIAGNOSIS IN X-RAY IMAGES USING CUSTOMIZED NETWORK