Self-Assessment of Having COVID-19 With the Corona Check mHealth App

At the beginning of the COVID-19 pandemic, with a lack of knowledge about the novel virus and a lack of widely available tests, getting first feedback about being infected was not easy. To support all citizens in this respect, we developed the mobile health app Corona Check. Based on a self-reported questionnaire about symptoms and contact history, users get first feedback about a possible corona infection and advice on what to do. We developed Corona Check based on our existing software framework and released the app on Google Play and the Apple App Store on April 4, 2020. Until October 30, 2021, we collected 51,323 assessments from 35,118 users with explicit agreement of the users that their anonymized data may be used for research purposes. For 70.6% of the assessments, the users additionally shared their coarse geolocation with us. To the best of our knowledge, we are the first to report about such a large-scale study in this context of COVID-19 mHealth systems. Although users from some countries reported more symptoms on average than users from other countries, we did not find any statistically significant differences between symptom distributions (regarding country, age, and sex). Overall, the Corona Check app provided easily accessible information on corona symptoms and showed the potential to help overburdened corona telephone hotlines, especially during the beginning of the pandemic. Corona Check thus was able to support fighting the spread of the novel coronavirus. mHealth apps further prove to be valuable tools for longitudinal health data collection.


I. INTRODUCTION
A T THE beginning of the COVID-19 pandemic, there was a lot of uncertainty about the novel coronavirus SARS-CoV-2 and knowledge about it was sparse. Quickly, healthcare systems were overstrained, and telephone hotlines overburdened. There was a huge demand for getting a quick first assessment about the probability of being infected and support in case of a possible infection. We designed and developed an mHealth (mobile health) app called Corona Check that allows users to answer a questionnaire to get a first assessment about their symptoms/situation with regards to being infected with SARS-CoV-2. We released Corona Check on April 4, 2020, on Google Play and the Apple App Store. As of October 30, 2021, there were almost 90 thousand assessments from more than 50 thousand users. While similar systems have been proposed before [1], [2], [3], to the best of our knowledge, we are the first to report about a large-scale deployment.
The first main benefit of our system was that it gave users immediate individualized feedback and behavioral recommendations based on their symptoms and contact history. Moreover, the users received general hygiene behavior tips. The system thus alleviated pressure from, e.g., telephone hotlines. The second main benefit of Corona Check was that, with the consent of the users, we were able to collect age-and sex-specific data for research about the occurrence and regional spread of corona symptoms.
Our main contribution is the introduction of our system Corona Check, highlighting how a quickly developed mHealth system aimed at the average user can support vast amounts of users in the beginning as well as during the COVID-19 pandemic. In Section II, we give an overview about related work. In Section III, we present the technical details of Corona Check, including the user perspective as well as the system architecture and special requirements for apps in the medical field. Section IV presents an overview of the data collected so far and age-and sex-specific results. In Section V, we discuss the limitations of our system. In Section VI, we discuss our results, draw conclusions, and point out future work.

II. RELATED WORK
Information technology and methods from computer science have been used from the very beginning of the COVID-19 pandemic in a variety of ways and for a variety of purposes. With vast amounts of data being quickly available, artificial intelligence and especially machine learning are often applied in COVID-19-related research. In their survey paper, Khan et  COVID-19, and drug research [4]. Research related to COVID-19 that is employing machine learning, for example, is about analyzing x-ray images [5], [6], [7], [8], [9], [10] and cough sounds [11], [12], or about making predictions about the spread of the virus [13], [14], [15]. What the mentioned research has in common is that the users of these systems were doctors, epidemiologists, researchers etc. -but not laypersons from the general public.
In contact tracing apps, the average user interacts with a system related to COVID-19. After Alice has been in close physical proximity with someone called Bob who later tested positive for SARS-CoV-2, typically measured by Bluetooth signal strength of smartphones nearby, she can receive a notification and get herself tested. Software system architectures, privacy concerns, and public perception of contact tracing apps have been discussed extensively [16], [17], [18], [19].
Another related category of apps is that of symptom tracking. Menni et al. reported about an app that tracks potential symptoms [20]. A fraction of the app users has undergone a COVID-test, revealing that loss of smell and taste was higher in those who tested positive. Klaser et al. [21] used the same app for tracking levels of anxiety and depression in the U.K., finding small associations between SARS-CoV-2 infection and anxiety and depressive symptoms. An updated version of the app was used to track symptoms of infected people to compare symptoms between the delta and omicron variants of SARS-CoV-2 [22].
Much less attention was spent on mHealth or expert systems that regular users can use in order to get a first feedback based on their own symptoms. Especially in the early phases of the pandemic, there was a high level of uncertainty in the population about how to behave if there was a suspicion of infection. At this stage, expert recommendations were offered by telephone hotlines from the public health system. These recommendations were primarily based on symptoms, contact history and travel history, as laboratory tests were not yet widely available.
An app-based expert system which provides individualized recommendations based on symptoms, contact and travel history has many advantages: It can provide important individualized information in an efficient way, can be easily updated in the rapid changing situation (e.g., the emergence of new high risk areas) and scales easily. By providing easy access to an accurate and up-to-date individualized recommendation, an app-based expert system can importantly contribute to inform the population how to best behave in order to protect themselves and others, which is an essential aspect in the early management of the pandemic. Moreover, the app-based expert system has the potential to relieve pressure of overloaded telephone hotlines or overwhelmed experts.
At the very beginning of the pandemic (publication in March and April 2020), two papers were published sketching the idea of mHealth expert systems for the diagnosis of infections with the novel coronavirus [1], [2]. Both papers present PC software prototypes based on the idea of having a rule-based system and checklists of symptoms that the user fills out. There are no reports of deployment nor detailed evaluations of the developed prototypes.
Hakim et al. also developed an Android expert system for diagnosing COVID-19 based on a rule-based system [23]. The input is a questionnaire about symptoms and travel and contact history. The authors reported about a small user study with 12 participants.
Banjar et al. reported about the development of a prototype of a COVID-19 diagnosis and management expert system [24]. The target audience are doctors in Saudi Arabia. The expert system handles the patients' data, their Electronic Health Records (EHRs), and processes current COVID-19 guidelines in order to classify the patients by their medical condition.
Mufid et al. developed an Android app that consists of an expert system for early detection of having COVID-19 and an information module that displays current news about the spread of the virus [25]. The app was tailored specifically for Indonesia. The expert system is based on a 16-item questionnaire about symptoms and contact with infected people. The system returns a risk status from "very low," over "medium," to "high risk." The authors reported about a usability study with several participants (exact sample size not specified).
A related field to expert systems is chatbots. Battineni et al. developed a chatbot asking the user about symptoms and refer the user to a doctor if a certain threshold is met with the answers. In their evaluation, the authors compared their approach to other existing chatbots. Erazo et al. developed a web-based chatbot to alleviate the pressure on the health care system [3]. A smallscale user study (exact sample size not specified) showed that the users found the system useful. Almalki et al. cover more about chatbots related to the COVID-19 pandemic in their survey paper [26].
There are some works on using wearables or other small devices for trying to detect SARS-CoV-2 infections. Mukhtar et al. developed a device based on Arduino hardware that measures heartbeat, cough severity, temperature, and blood oxygen level for detecting COVID-19 [27]. In contrast to questionnaire-based solutions that the average user can perform on his/her smartphone, this solution is rather targeted at use in hospitals or to monitor patients at home. Astriani et al. presented a smart mirror measuring heart rate and temperature in order to warn about possible infections [28].
There is some work that proposed designs, frameworks, or methodologies for systems that detect an infection with the novel coronavirus. Skibinska et al. proposed a methodology for early-stage detection of COVID-19 based on data from wearables [29]. Maghded et al. proposed a design for a framework that uses the smartphone's sensors to detect an infection with the coronavirus [30]. They proposed using a variety of sensors from the smartphone, including, e.g., measuring the temperature or taking photos of CT scan images of the lung. Belkacem et al. proposed a hypothetical end-to-end-pipeline for detecting different respiratory infections [31]. What these approaches have in common is that they all rely on study results and/or machine learning models that contain knowledge about SARS-CoV-2 infections gained during the ongoing pandemic. Similarly, Li et al. developed a mobile system capable of analyzing x-ray images of COVID-19 patients [32]. Imran et al. developed an app that records coughing sounds, analyzes them in the cloud, and returns a preliminary diagnosis of COVID-19 [33].
Overall, COVID-19 related research is being conducted in many directions. The amount of related work specifically in the domain of mobile mHealth systems or expert systems is sparse. What the existing related work in the domain has in common is that a lot of the work was preliminary. To the best of our knowledge, we are the first to report about a large-scale deployment of a mobile system in this context.

III. TECHNICAL DETAILS
In this section, we present the technical details of Corona Check. Corona Check is based on the TrackYourHealth platform and API [34], [35], [36], [37]. The TrackYourHealth platform proved as a valuable base for several other questionnaire-based health-related apps [38], [39]. The backend is based on PHP and serves to native mobile apps for Android and iOS. For technical details of TrackYourHealth, please refer to the cited papers. In this work, we highlight the parts that are specific to Corona Check. In Section III-A, we show Corona Check from the user perspective, while in Section III-B, we detail the core functionality of Corona Check and the feedback system that gives the user immediate feedback after filling out the COVID-19 evaluation questionnaire. Section III-C describes the tips module of Corona Check, which provides the user with additional information about the ongoing pandemic. In Section III-D, we give an overview of the data we collected, and in Section III-E, we briefly highlight the specific requirements that were necessary for publishing an app related to the coronavirus.

A. User Perspective
Corona Check was released on Google Play and the Apple App Store in German and English on April 4, 2020. Fig. 1 shows the general user journey when using the app. (1) The user can start a questionnaire that is filled out for himself/herself or another person. This is shown in Fig. 1 on the top left. (2) After starting the questionnaire, the next screen will ask for additional information, for example, as shown in the figure, information about travels. (3) Next, the questionnaire screen is shown where the user answers COVID-19-specific questions and demographic questions. If the user agrees, a coarse location (to protect the privacy of a user) of the device will be stored. Table I   Corona Check has two additional features besides the check itself. The Tips section (top right of Fig. 1) contains health-related tips and recommendations for daily life during the pandemic, e.g., on hygiene. Each tip can be rated with 1 to 5 stars and the average rating is displayed for each tip. The News section (bottom right of Fig. 1) displays the latest news about the ongoing pandemic.

B. Feedback System
We developed the questionnaire and rule-based feedback system with the medical and public health experts in our team (including the Bavarian Health and Food Safety Authority).
Note that the list of symptoms (Question 1 in Table I) is based on the available knowledge at the time. Diarrhea was an additional symptom option to choose during the first two months Corona Check was online. Questions 3, 4, and 5 on age, sex, and education served two purposes for future studies with the Corona Check data. Firstly, these variables can be used to check how representative the Corona Check user group is of a general population. Secondly, we can analyze COVID-19 while controlling for these variables. Educational level was used as a proxy for social status, which has been shown to influence health [40].
The feedback and tips were developed in accordance with the recommendations by the German federal agency for disease control and prevention (RKI, Robert Koch Institute). Overall, there are four possible outcomes bound to the current symptoms and the past contact with infected persons (note that Corona Check was released prior to the availability of vaccinations): (a) no symptoms and no contact, (b) no symptoms and at least one contact, (c) at least one symptom and no contact, (d) at least one symptom and at least one contact (also see Fig. 1). The color-coding of the results immediately indicates the level of concern. Each result gives a detailed answer about the situation with advice for the next steps. The feedback algorithm lies at the core of the Corona Check expert system. It fulfills the role that typically humans fulfill at the end of a telephone hotline. Corona Check asked the questions that the human operator would and gives the advice that the person would.

C. Corona Tips
Corona Check users were provided with 30 different tips 1 on how to safely deal with the pandemic. The 30 tips have been developed by our medical and public health experts. For example, they explain the importance of proper hand washing or complying with contact restrictions. Besides explaining the importance, the tips also contain practical advice for concrete behavior. The tips were displayed unrelated to symptoms entered by the user. All tips were displayed for all users.

D. Database
The user-generated as well as the operational data of the server application are stored and managed in a relational database, since data integrity and consistency mechanisms (e.g., constraints and ACID transactions) are integrated and well-tested. Moreover, it is easier to create and maintain highly interrelated data models with this type of database system. In addition, relational databases provide a sophisticated query language suitable for complex analytical queries required for data evaluation. To give insights into the project's database schema, Fig. 2 depicts an Entity-Relationship Model (ERM) of the Corona Check database using crow's foot notation. Note that Fig. 2 illustrates an excerpt of the entire database schema. The selection represents the core entities containing most of the user-generated data. Due to the multilingual project setup, the database schema contains several entities with translated text data, marked with a T , which are omitted in the ERM to further simplify the model.
To describe the data set in this work, a snapshot was extracted on October 30, 2021. Corona Check combines the ideas of mobile crowdsensing [41], [42], [43] and Ecological Momentary Assessments [44] to collect user data. For this reason, the entity Users constitutes a central and high-related table with 145,223 verified users. In addition, the users' actions (e.g., questionnaire filled out) are stored in the table User Histories to provide insights into the usage of Corona Check. Technically, Corona Check could contain multiple different studies (i.e., (sets of) questionnaires). Each study, in turn, can have so-called collaborators managing the study's content. A collaborator is a user with additional (role-based) permissions on specific studies. Corona Check only contains one study, containing the questionnaire given in Table I. All users are associated with this one study.
To collect data in a structured way, one study has one or more questionnaires in the entity of the same name. Each questionnaire can be versioned. The Corona Check study contains one questionnaire, available in each supported language. Questionnaires are structured with polymorphic buildings blocks called Questionnaire Elements. Elements can be of the type Page Element, Text Element, Question Element, Headline Element or Media Element. These building blocks can be used to create complex medical or psychological questionnaires meeting requirements for a variety of studies. The submitted answers for a questionnaire are serialized and stored in JSON in the entity Answer Sheets (n = 86,912). The entity also stores sensor data (e.g., location) and client device information (e.g., operating system) in the same table.
In addition, Corona Check provides tips (see Section III-C). Tips can also be managed and stored in different languages. A like feature allows users to rate the provided Corona tips. Tips were rated by 970 unique users. In order to give the user feedback immediately after submission of a questionnaire, such a questionnaire may reference to one or more key-rule pairs that are stored in the entity Feedback. Rules, evaluated on the client side, can be managed and adjusted dynamically.

E. Medical Device Regulation
Recently, the requirements specifically for mHealth-related (mobile health) apps have been increasing. When we released Corona Check, we had to comply with the medical device regulation (MDR). With the MDR, strict rules have to be met regarding the validation and documentation of each software module. Especially in situations like the beginning of the coronavirus pandemic, when software solutions were required in as short a time as possible, such requirements can pose a risk for timely app releases. For more details refer to our publications related to the topic [37], [38], [45]. We were among the few apps that adhered to the MDR in the beginning of the COVID-19 pandemic with our mHealth system Corona Check. On top of the medical device regulation, the app stores of Google and Apple were especially cautious about allowing apps related to COVID-19 into their stores, likely to prevent allowing malicious apps. Overall, when releasing Corona Check, complying with all necessary regulations took time before the app could be found by the average user. Additionally to all the regulations mentioned above, this study was approved by the ethics committee of the University of Würzburg with ethical approval no. 71/20-me on April 4, 2020.

IV. RESULTS
To show that our mHealth system is a feasible solution that is able to reveal meaningful results, we present selected results using descriptive statistics. Therefore, we analyzed the users' self-reported data. Overall, at the time of data extraction, from all 145,223 users in the database, 52,267 (36%) filled out at least one assessment. From those 52,267 users, there were 86,912 assessments. To obtain the final dataset, we filtered the data set twice. First, we removed all assessments without a research release. Users could explicitly state if they agree to their data being used for research purposes (see question 7 in Table I). This leaves us with 56,655 remaining assessments (65.2%). Second, we removed implausible assessments as follows: Each time, the questionnaire is filled out, the user enters age range and sex (questions 3 and 4). We classified an assessment as implausible if a user repeatedly filled out the questionnaire for himself/herself, but either the age or the sex differed from the first time he/she filled out the questionnaire.
The final dataset had 51,323 remaining assessments from 35,118 users, stemming from a total of 140 countries. Most assessments were filled out for the user himself/herself; 4,741 users (13.5% of all users) filled out 5,877 assessments (11.5% of all assessments) for others. For 36,212 assessments (70.6%), the users shared location information with us. Overall, 47,066 assessments (91.7%) were conducted with an Android device, the rest with an iOS device. Looking at multiple assessments, we observed that 80% of users filled out only one assessment. The remaining 20% (7,075 users) filled out 3.29 assessments on average (SD = 9.58) and the mean timedelta between first and last assessment was 18.88 days (SD = 55.43). Fig. 3 shows the number of all assessments over time. There were more assessments in the earlier phases of the COVID-19 pandemic. The plot shows two peaks for June 2020 and September 2020. We added the number of confirmed new cases for Germany, India, South Africa, and the world in the plot [46]. 2 In Table II, we present details about the demographic information of the included users. We present the global completion behavior of assessments in Fig. 4. It shows the number of assessments completed per country.
The distribution of the reported symptoms between groups did not differ significantly when stratified for age group and sex (Table III). We also looked at the distribution of symptoms between countries. We only investigated countries with at least 51 users. An ANOVA test could not detect statistically significant differences in symptom distributions between countries. However, the number of reported symptoms differed between countries. Users from Inida reported more than twice as many symptoms per assessment as those from Germany (2.81 vs. 1.27). Users in France reported the fewest symptoms at 0.38 per assessment. Users from Uganda reported the most symptoms at 4.03 per assessment. The average number of reported symptoms per assessment per country is shown in Table IV . Although the completion of the questionnaire differed between countries, the distributions of symptoms seemed to be rather similar.
Corona Check overall contains 31 general tips on hygiene, see Section III-C. A total of 3,538 ratings were submitted with an average rating of 3.7 out of 5 stars (SD = 1.67). The top-rated tips were (in descending order): (i) protecting wounds, (ii) how to behave in daily life, (iii) when to wash hands, (iv) masks that cover the nose and mouth, and (v) handling surfaces and objects. The following tips received the lowest ratings: (i) How to cover sneezing or coughing, (ii) tissues, (iii) smear infection, (iv) shaking hands and hugging, and (v) traveling. An overview of all tips and the distribution of their ratings is given in Fig. 5. We further analyzed who rated the tips. Table V shows the age distribution for all users compared to those who rated at least one of the tips. We found that older users rated more often than younger users. While the most common age group for all users was 20-29 years, the one for users who rated the tips was 60-69. Table VI shows the location distribution of all users compared to those who rated the tips. The results show that users in Germany were largely overrepresented in the group of users who rated the tips.

V. LIMITATIONS
Several limitations of Corona Check have been revealed during its practical use. We quickly encountered the need to adapt the questionnaire and its feedback texts in frequent cycles as an important feature due to changing recommendations and new findings regarding COVID-19-related symptoms. However, providing a robust mechanism that does not confuse or distract users with respect to different questionnaire versions and frequent changing feedback texts is important, but not simple. As any change to the app must be considered in the context of the medical device regulation (and time matters during COVID-19), we decided to show only the most recent version of the questionnaire as well as the most recent feedback texts, which mitigated measures for the medical device regulation and saved us time.
Although we had not complaints about our approach, possibly, a more fine-grained approach might fit the users' needs and the phases of the pandemic better.
In addition to the provided questionnaire, we quickly saw the need to display further information in a smart way when filling out the questionnaire. For example, by the time certain regions have been declared risk regions, it was important to   update this information as well as letting travelers know about possible consequences when traveling to or returning from these regions. However, the provision of the information was necessary in a way that the existing questionnaire-procedure can be distinguished from this new information.
Another limitation that we encountered was that for users who filled out questionnaires multiple times, it had to be checked, whether all completed questionnaires can be used for evaluation or if the users just wanted to 'play' with all the combinations of filling out the questionnaire to see what feedback is possible. The gambling behavior, in turn, might affect the validity of the data. Corona Check was provided in German or English. More languages could increase its use in more countries and multilingual societies and thus increase the number of filled-out questionnaires. This may be helpful for better insights into the development of the pandemic in a rather short period of time.

VI. DISCUSSION
To the best of our knowledge, we are the first to report about a large-scale deployment of a mHealth system for assessing potential COVID-19 symptoms. Corona Check provided specific symptom-related advice as well as general tips for behavior and hygiene. We highlighted the technical details of Corona Check and analyzed the collected data.
First of all, we note that Corona Check was not as widely known as some contact tracing apps, which received adoptions rates as high as 50% of the population (Germany) [18]. We did not advertise extensively for Corona Check, and self-assessment apps did not received as much media coverage as contact tracing apps.
We found that only 36% of the users filled out at least one assessment. Out of these, 80% only filled out one. There could be several reasons for this. Maybe the news and tips sufficed for many users' purposes. Maybe users just wanted to see what the app does and then decided to not use it or to use it only once. Overall, we found that more younger users used our app (see Table II); most users are below 40 years of age. This is in line with the idea that younger users tend to be more tech-savvy and more likely to use an app instead of calling a hotline. For 65.2% of all Corona Check assessments, the users agreed to their data being used for research purposes. In the final dataset, we had geolocation information for 70.6% of the assessments. Thus, in line with some of our previous studies, we found that most users are willing to share their data with researchers [47].
We did not observe that the number of new confirmed cases influenced the number of Corona Check assessments (see Fig.  3). Likely, the two peaks in Corona Check assessments in June 2020 and September 2020 are due to some news or social media posts creating a brief period of increased public interest in Corona Check. A broader active advertisement of mHealth systems like Corona Check might create a larger user base. Then, we would expect to see some correlation between in-app-assessments and new cases. After the two peaks, we observed a steady decline in the number of assessments per day. There could be two reasons for that. First, existing users might lose interest in the app and fewer new users were onboarding. Second, with passing time, public knowledge about corona increased, as well as the availability of testing stations, minimizing the need for an app like Corona Check. Thus, our user data strongly supports the notion that an app-based mHealth system for the population is particularly important in the early stage of a pandemic. Note that testing was not widely available during the beginning of the pandemic and Corona Check did not offer to register test results. Hence we were not able to investigate to what extent the high risk warning indicated a real infection.
Regarding the symptoms entered in Corona Check, we did not find statistically significant differences between age groups, or between countries. This may indicate that symptoms were independent of these variables. Regarding the general hygiene tips in Corona Check, overall, the ratings indicated that they were perceived as helpful, see Fig. 5. We observed that the proportion of older users rating the tips was higher than the proportion of younger users (see Table V). Raters from Germany were disproportionately overrepresented among the raters of the tips (see Table VI). We presume that users in Germany might have been aware that Corona Check was made in Germany, leading to higher identification with the app or trust in the app, and thus, a prolonged usage including rating the tips.
Overall, we have shown that an mHealth system such as Corona Check can help support much of the functionality that a telephone hotline by, e.g., authorities or health insurances, would serve. With increasing public knowledge about symptoms related to the new virus and broadly available testing stations, the need for an mHealth system for detecting coronavirus infections might be reduced. Thus, especially during the early phase of the pandemic, Corona Check was a valuable contribution in fighting the global COVID-19 pandemic.