A Review on the Mobile Applications Developed for COVID-19: An Exploratory Analysis

The objective of this research is to explore the existing mobile applications developed for the COVID-19 pandemic. To obtain this research objective, firstly the related applications were selected through the systematic search technique in the popular application stores. Secondly, data related to the app objectives, functionalities provided by the app, user ratings, and user reviews were extracted. Thirdly, the extracted data were analyzed through the affinity diagram, noticing-collecting-thinking, and descriptive analysis. As outcomes, the review provides a state-of-the-art view of mobile apps developed for COVID-19 by revealing nine functionalities or features. It revealed ten factors related to information systems design characteristics that can guide future app design. The review outcome highlights the need for new development and further refinement of the existing applications considering not only the revealed objectives and their associated functionalities, but also revealed design characteristics such as reliability, performance, usefulness, supportive, security, privacy, flexibility, responsiveness, ease of use, and cultural sensitivity.


I. INTRODUCTION
C ORONAVIRUSES (CoVs) are a family of viruses that can infect human beings as well as different species of animals like cattle, cats, and bats [1].These viruses can cause illness varying from the very mild cold to severe health conditions including death.A pneumonia, referred to as Severe Acute Respiratory Syndrome (SARS-CoV), was first recognized in China [2] with an overall death rate of around .3 to 6% [3].Like SARS-CoV, Middle East Respiratory Syndrome (MERS-CoV) is also a member of CoVs family.And, the 2019 coronavirus (COVID-19) is a new betacoronavirus, which has infected humans recently.Common symptoms of COVID-19 virus include fever, cough, and breathing difficulties, among others.In severe cases, the infection can cause pneumonia and even death.People with previous diseases related to heart, kidney, lungs, and diabetes are more likely to develop serious illness from COVID-19.After China, COVID-19 has been spread in more than 210 countries and territories [4].On March 11, 2020, the World Health Organization (WHO) has declared COVID-19 as the global pandemic [5].
The immediate risk of being infected by COVID-19 is increasing day by day as the outbreak expands.Since there had been limited or delayed precautionary responses against the new virus, it had spread rapidly from person-to-person globally.It has been suggested that further COVID-19 spread can be controlled by taking proper and urgent precautions and raising health awareness among people [6].Several International organizations like WHO [7], Centers for Disease Control and Prevention (CDC) [8] have suggested necessary plans to deal with the pandemic.Countries have taken steps such as providing health safety guidelines, imposing travel restrictions, organizing enough quarantine and hospital facilities for infected people, and even complete lock-down [9] [10] [11] [12].To prevent virus infection some suggestions include: frequent hand washing, avoiding crowded places, covering mouth and nose, eating fully cooked meat and eggs, and avoiding close contacts with anyone coughing and sneezing, among others [1].Some information and communication technology (ICT) based initiatives have been taken for raising awareness for preventing the pandemic caused by COVID-19.These in-M.N. Islam et al.: A review on the mobile applications developed for COVID-19: An exploratory analysis clude mobile applications, online dashboards to provide live updates, websites to provide remote assistance, web portals to provide useful information related to COVID-19, and the development of electronic gadgets to detect or scan the symptoms of COVID-19, among others [13] [14] [15].
At present, more than 45% people in the world use smartphones [16].Indeed, smartphones and Internet penetration in the developed world is high.At the same time, recent studies show that the uses of smartphones and the Internet have also been noticeably high in many developing countries [17].As an example, a study conducted by Islam et al. [18] found that around 234 mobile health (mHealth) applications were developed explicitly for Bangladesh.Developing countries have relatively weak healthcare facilities.Therefore, there is a possibility that COVID-19 may hit hard in these countries.Mobile apps can support interactivity, visual and auditory content, real-time data collection, as well as links to social functionalities [19] [20].Considering the widespread adoption of smartphones in the developing and developed countries, there is a growing opportunity for the effective uses of mobile applications for preventing, caring and controlling the pandemic caused by COVID-19.
A number of review studies were conducted focusing on the mobile health applications developed for cardiovascular disorder [21], depression [22], human immunodeficiency virus [23] [24], cancer [25], schizophrenia [26], monitoring physical activity [27], and managing bipolar disorder [28], among others.However, up to date, no study has assessed the available apps related to the COVID-19 pandemic.Indeed, as the COVID-19 pandemic is a newly developed phenomenon and therefore, available apps in this area are still limited and more efforts in app development will follow in the future.However, at the same time, we believe that as it has become a worldwide pandemic, research needs to investigate what apps have been developed and make a synthesis to guide future service development.
In order to address the above-mentioned research gap, this study conducts a thorough review of the existing mobile apps that have been developed focusing on COVID-19.Our objectives were to (i) identify existing apps related to the COVID-19 in the popular app stores; (ii) investigate the apps in terms of their objectives and associated features or functionalities; (iii) explore the platform (operating system) of the apps, country-context for which the app was developed, and the language of the app; and (iv) identify important design characteristics based on the positive and negative feedback from users.

II. METHODS
Reviewing the existing apps developed for COVID-19 was conducted by a systematic app review.The study was carried out following the steps discussed in the following subsections.

A. SYSTEMATIC SEARCH CRITERIA AND SELECTION
A systematic review of mobile applications across three major mobile app stores: Android Google Play, Apple Store, and Microsoft Store was conducted on 30th April 2020.The search keywords used to find out the related applications were: "coronavirus," "COVID 19," "corona," "corona outbreak," "corona pandemic," "corona and symptom monitoring," "corona and self-care," "COVID 19 and symptom monitoring," "corona and COVID 19," "corona and self-isolation," "corona and quarantine," and "self-care and COVID 19."Each term was searched in each of the three app stores listed.
App titles, screenshots of the apps, and app descriptions were considered for the preliminary screening.The second round of exclusion criteria focused on removing the duplicate apps that were found in multiple stores or from multiple search terms or the apps that were no longer available (some of the apps were removed from the app stores).Two members of this research team meticulously reviewed the apps in each round of exclusion.Finally, a total of 25 apps were selected, downloaded, reviewed, and evaluated.The description of the selected applications is provided in Table 1.The related app screening, inclusion/exclusion, and selection process are shown in Figure 1.

B. EXTRACTING THE STUDY DATA
To attain the research goal, we extracted data related to the objective of the apps, functionalities provided by the apps, the platform (operating system) of the selected apps, countrycontext, language of the apps, user ratings, first release, and user comments.The objectives and features of the apps were extracted both from the app description and by the experimental use of the apps.For extracting the features, we first went through the app description and kept notes of the stated features.In some cases, when the app language was not English, we used Google translator.After that, each app was installed and explored to re-check (verify) the extracted features and update the list of extracted features as observed from the experimental use of the app.We ensured that the features extracted against each app from the description must be present in the actual app through the experimental use of the app.Our approach helped us to explore the stated features as well as other features (if any) from our experimental use of the apps.

C. DATA ANALYSIS
An affinity diagram [29] approach was used to analyze the data to find out the objectives of the apps.A noticingcollecting-thinking approach [30] was used to analyze the user comments (n = 1574) of the selected applications to identify the design characteristics of the apps.The other data including the review scores, development platform, application languages, and the country-context were analyzed using descriptive statistics and data synthesizing.We note that our approach of data collection from online sources is widely used in different disciplines [31]- [33], and known as netnography [34].
Each of the selected apps was investigated to extract their functionalities or features.Each author of this article separately participated in exploring the app functionalities and then in grouping (mapping or clustering) process through an affinity diagram [29].Firstly, each author extracts the functionalities and assigned names to the functionalities.After that they met together to compare and verify the extracted functionalities for each application.The naming of the extracted functionalities, conflicting functionality names, and disagreements were resolved by discussion through consensus to finalize the functionalities of the selected apps.Secondly, each researcher separately categorized the extracted functionalities into different groups based on their similarities.Next, an intuitive name was given to each group (which are considered as the objective of the application) and then drew the affinity diagram.Finally, the four affinity diagrams (by four authors) were analyzed together to review the grouping or clustering.Thereafter, we came up with one affinity diagram with the final clustering.
Since the user review data are qualitative in nature and not all the reviews were related to the design issues, we have followed the noticing-collecting-thinking [30] method to analyze each of the retrieved review comments.An example of analyzing a review comment through the noticing-collectingthinking approach is discussed here.Firstly, at the noticing level, we read a review comment several times to notice an interesting or relevant aspect that may relate to the design of the application and coded it accordingly.For example, the following user review comment was noticed, which was relevant to the application design: "I did not like that you have to sign up with your phone number before you can enter the app.I am concerned about digital privacy and this is just not okay, especially in this crucial time when information is crucial." The extracted code of this review comment was 'concerned about digital privacy'.Secondly, in the collecting level, all the related codes were grouped and given a self-expressive or self-descriptive group name.The 'concerned about digital privacy' code was grouped under the category of 'Privacy'.Finally, in the thinking stage, we examined the things (e.g., all codes related to 'privacy') that we had collected to verify whether the things make some type of sense (e.g., a design characteristic or recommendation) out of each collection.Furthermore, we checked whether any relationship exists within a collection and across the collection.While collecting and thinking, a review comment was coded as either positive or negative based on its referential or indicative meaning.For example, if any user review expresses the difficulties of using a feature or reveals dissatisfaction with the app (e.g., providing unauthentic and outdated information), then we have considered such a statement as a negative comment.The example quote stated above expresses dissatisfaction of the user as he/she was asked to sign up using a phone number and therefore, was concerned about digital privacy.Thus, the above comment was coded as a negative comment.
The review comments were independently analyzed by two researchers.The comments were meticulously read to notice, and code as we discussed above.The assigned key-FIGURE 2. Mind map of the features and objectives of the apps words or codes were then collected and categorized to represent common themes.After that, both researchers thought about their coding to refine, verify and update the code names, categorizations, relations among the categorizations.Finally, all authors met to compare the coding and classifications.The inter-coder agreement [35] was 0.91.The disagreements were resolved by discussion through consensus.Using an iterative process, all researchers read, sorted, reread, and recombined the data until consensus was achieved.

III. RESULTS
This section discusses the findings of reviewing the selected mobile applications.

A. APP OBJECTIVE AND FEATURES
As an outcome, a total of 26 functionalities were found from the 25 mobile applications.The extracted functionalities for each mobile application are shown in Table 3 of Appendix A. Through the affinity diagram, a total of nine objectives were found for which the mobile applications were developed.The extracted functionalities and their mapping to the objectives of the mobile applications are presented in Figure 2. The objectives of the mobile applications were also mapped to the selected mobile apps and shown in Table 4 of Appendix A. The results (see Table 4) show that most of the applications were developed for the multiple purposes but there was no single application found that addressed all objectives or purposes.The number of applications that offered a specific objective is shown in Figure 3.The revealed purposes of the mobile applications for the COVID-19 pandemic are briefly discussed below.2) Monitor patients -Around half of the applications (i.e., 12 out of 25 applications) include features for monitoring patients by observing the users' health parameters such as temperature, heart rate, oxygen, and blood pressure using Bluetooth certified medical devices.The patients can consult with doctors through live video and receive medical advice.Some apps provide virtual assistance through interacting with people in natural language and voice.3) Current status -Most of the applications (i.e., 18 out of 25 applications) provide the current status and statistics from a local (specific-country context) and a global perspective through data-sheets and data visualizations.Such status includes the number of confirmed infected people, recovery, death, incubation period, etc. with respect to the time and geographical location.Some apps provide the prediction data through different kinds of mathematical analyses; and some other apps provide recent corona virus related news and push notifications for any new suspected case, suspected community/regions, death, and recovery.to take necessary initiatives to prevent or control the spread of COVID-19.7) Treatment services -Around half (48%) of the applications provide treatment services or treatment-related information.The available functionalities to achieve this objective are: search functionality to find nearby medical facilitates with COVID-19 testing capacity and pharmacies; provide information about the nearby hospitals, hospital maps, and travelling information to reach the nearby hospitals.Some applications provide information related to COVID-19 symptoms so that in case of suspected infection, users may conduct the selfassessment.Some other apps recommend suspected patients to contact the nearby service centers or hospitals based on their geographical location and physical symptoms.8) Raise awareness -Similar to the current status obmost of the applications (i.e., 17 out of 25 applications) make people aware of the pandemic by providing the recent updates (corona news and corona status).In addition, some applications provide useful web links related to the COVID-19, information about when, how, and why to stay at self-isolation.A few apps provide declarations about new suspected cases, death, recovery, etc. from a local (a specific country) perspective and the push notification for any important/emergency alert message and news about suspected community/regions.9) Improve mental health -A total of seven (28%) applications focused on users' mental health which is very crucial for each individual during the pandemic.These applications improve motivation and self-confidence of the users through community building, sharing personal experiences with other people, and by providing the positive COVID-19 news like recovery updates of the patients and recent/new initiatives taken by the government in order to prevent and control the spread of COVID-19.

B. APP CONTEXT, LANGUAGE AND USER RATINGS
The summary of the evaluated apps as presented in Table 1, shows that among the existing applications, 4 apps (16%) were developed exclusively in iOS platform, two apps (8%) were developed in Windows platform, 12 apps (48%) were developed in Android platform and the remaining apps were developed in both iOS and Android platforms.All the apps were freely available.Each application mentioned about its privacy policy.More than half of the applications (n=13) were developed for a specific country context that includes India, Vietnam, Brazil, France, Bangladesh, UK, Canada, Spain, Australia, and Greece.Among the apps, twelve were developed in local language and one app COVA Punjab was in English (see Table 1).

C. MINING THE APP REVIEW
A total of ten information systems design characteristics were found on the mobile applications developed targeting COVID-19 through mining the app reviews.Both positive and negative experiences were found in each category of design characteristics.The design characteristics codes and the examples of users' comments are presented with the + (positive) and -(negative) signs in Table 2.The design characteristics related factors (see Table 2) are discussed below.
Reliability refers to the dependability of the service [36]- [38] such that it enables users to receive authentic and updated information, and make sure that the app contents don't encourage panic but promote awareness.Performance captures the predictability and functionality dimensions [36].Therefore, performance relates to issues such as wellexecution of the app functionalities, help in slowing down the spread of the virus, and failure to access some of the app functionalities properly.Usefulness and Supportive are concerned with the extent to which an application helps the users to achieve their purposes such as raise awareness about the coronavirus, properly keep track of the updated situation, support for self-caring, and maintain social distancing.We note that the concept of perceived usefulness has been widely used in prior literature and has been suggested as one of the main predictors of adopting and using a service [39]- [41].Security and Privacy relate to the users' concern on digital privacy, accessing the application anonymously, unnecessary resource access permission, and asking for personal information or phone number during sign-up.Both security and privacy have been suggested important design considerations in prior literature [42].Flexibility refers to the way the system adapts to the changing demands of the users [36], [37].Therefore, in our context flexibility is about allowing the users to post without world limit, avoiding rigid information (e.g., phone number) input (i.e., in a specific data format), and allowing options to hide the posts and setting up reminders.Concerns related to receiving immediate feedback, activation code or OTP, and fast response/load are related to the responsiveness factor of information systems characteristics.The responsiveness characteristics can be captured through the timeliness and speed characteristics that have been discussed in prior literature [36].Ease-of-use and usable refers to the design aspect of user screens in terms of ease of use, visual attractiveness, user-friendliness, and convenience in delivering services [39].Therefore, these information systems design characteristics are concerned with the easy and convenient way to access information, use of intuitive navigation buttons and visual representations, and proper organization of the app contents (information architecture).Finally, the culturally sensitive factor refers to developing contextual (in terms of country and language) applications for more effective and efficient use of the app functionalities by the local people.We note that language has been emphasized by Wixom and Todd [37], whereas Zakaria et al. [43] emphasized on designing and implementing culturally sensitive IT applications considering privacy and cultural values.Each of the above information systems design characteristics related factors are mentioned both positively and negatively in the app review.For example, in the case of reliability factor (see Table 2), users' concerns related to providing authentic information, respectfulness in publishing information, and updating information in a regular basis were positive user experiences; while user comments related to providing insufficient information, spreading false information, discussing unrelated subjects and creating panic were negative user experiences.

IV. DISCUSSION
The review found that only a limited number of mobile applications have been developed.However, we note that more application development efforts will eventually come in the future.Our review revealed the main purposes of developing the mobile applications and the functionalities to achieve these objectives for the prevention, mitigation, and containment of COVID-19.The review study also explored the key factors or concerns that affect the end-user experience.Our results indicate that users prefer applications with higher reliability, performance, responsiveness, supportive, ease-of-use, usefulness, security, privacy, and flexibility.We also observed that culturally sensitive applications are needed.COVID-19 pandemic creates a crucial situation in the affected countries.

A. IMPLICATIONS
This research has implications for future research and practice.The outcomes of this review will greatly contribute to the health institutes, health workers, practitioners, and the governments of the COVID-19 affected countries.Our paper summarizes the currently developed apps to raise awareness about the existing mobile applications, their functionalities, and design characteristics.The findings can help to take necessary initiatives in developing new applications or updating the existing applications to receive maximum benefits out of these applications during the pandemic.The review study also can be considered as a requirement elicitation study.The app developers may consider the revealed objectives and functionalities as the user requirements.For example, if the professionals or governments want to develop a mobile application to ensure population awareness about the COVID-19 pandemic, they may incorporate the services associated with the 'raise awareness' objective.In contrast, if a government would like to develop a new application to provide all possible services related to COVID-19, it may consider the nine objectives and the entire set of functionalities revealed from this study as shown in Figure 2.
This review also helps to understand the key information system design characteristics that need to be addressed to develop such applications.The revealed design characteristics can be considered as the design recommendations to the practitioners for developing and evaluating such appli-cations.The design recommendations based on the revealed design characteristics are as follows.The applications should provide quick and accurate responses.The apps need to be usable and useful.The applications should provide authentic information in order to be reliable.The applications should also perform well with the required functionalities.This will also ensure that the applications are useful and supportive with respect to the user requirements.The security and privacy issues should be addressed properly since these aspects are critical to many users during the vulnerable time.Responsive and flexible app design and development need to be ensured.The application should be easy to use in order to ensure that different types of users can use it.Finally, the application should be culturally sensitive, when possible; since users may prefer to have a contextual (local) app in their own language.
In sum, the findings of our study would be a great source of inspiration for governments to take necessary initiatives to develop new applications and promote the adoption of existing/new mobile application(s).Finally, for app development companies, these outcomes provide an indication for developing new and innovative mobile apps targeting the affected countries.

B. LIMITATIONS
The study presented in this paper has a few limitations that are important to acknowledge.Firstly, the criteria (search strings/keywords) chosen for selecting the relevant apps for this study may not cover all the available applications, especially the applications that are named in a local language.Secondly, this study does not include the relevant apps that are developed or became available in the app stores after April 30, 2020.Thirdly, the review data was analyzed through the qualitative approach using the affinity diagram and noticing-collecting-thinking.Qualitative analysis is subjective and partly depends on analyzers' skills, expertise, and knowledge.Therefore there might be some flaws in the data extraction, coding, and clustering process.However, in order to alleviate these limitations, researchers meticulously conducted checks for apps selection and data analysis, and made adjustments where necessary through discussions.Fourth, in this study, we did not investigate which features are essential and which are nice-to-have.Thus, a future study may explore or classify the essential and nice-tohave features to achieve the identified objectives.Finally, the online review data collected from the app stores may contain biases.Therefore, future research can use multiple methods to collect and analyze data to investigate the validity of our findings. .

FrequencyFIGURE 3 .
FIGURE 3. The objectives of the mobile applications

4 )
Prevent COVID-19 -More than half of the selected applications (i.e., 15 out of 25 applications) provide services for the prevention of COVID-19 by providing updated COVID-19 related news about newly suspected community/regions.Especially, these applications provide the preventive guidelines suggested by the ministry of health, WHO, CDC, or other authentic body.Some apps focus to provide the necessary information for staying at self-isolation.5) Control COVID-19 -Around 40% of the applications (i.e., 10 out of 25 applications) target to control the spread of COVID-19 by providing instructions for self-isolation, facilitating to report the suspected cases to the health authority, nearest hospital, or to the government; and conveying the users feedback to the government bodies to take necessary initiative to control COVID-19.Some other applications provide information related to COVID-19 symptoms and keep people updated with the situation by providing updated news regularly, which also assists end-users to control COVID-19.6) Communication support -Around one-third of the applications (i.e., 8 out of 25 applications) include services to provide communication support using live chat or video consultation, instant messaging, regular or emergency phone calls to hot-lines/dedicated help numbers.A few applications provide interaction/communication facilities between the government bodies and people.The government can send notices, health recommendations, and record people's feedback This work is licensed under a Creative Commons Attribution 4.0 License.For more information, see https://creativecommons.org/licenses/by/4.0/.This article has been accepted for publication in a future issue of this journal, but has not been fully edited.Content may change prior to final publication.Citation information: DOI 10.1109/ACCESS.2020.3015102,IEEE Access M. N. Islam et al.: A review on the mobile applications developed for COVID-19: An exploratory analysis This work is licensed under a Creative Commons Attribution 4.0 License.For more information, see https://creativecommons.org/licenses/by/4.0/.This article has been accepted for publication in a future issue of this journal, but has not been fully edited.Content may change prior to final publication.Citation information: DOI 10.1109/ACCESS.2020.3015102,IEEE Access

TABLE 1 .
Summary of the reviewed apps related to COVID-19