Secure IoT Assistant-Based System for Alzheimer’s Disease

The increasing number of people with Alzheimer’s disease (AD) is a significant concern in many countries. Hence, new solutions for preventing, detecting and supporting persons with AD are required. The aim of this paper is to develop a prototype that provides psychological support services and ensures secure sending of information that can be investigated by a family member to protect the person with AD. The designed wearable prototype is able to classify the detected images into two categories including family/non family member based on a Convolutional Neural Network (CNN). Moreover, our prototype enables tracking the location of the person with AD. Furthermore, our IoT prototype protects the images captured by the webcam through the steganography technique that allows the recipient to decode the original image using a key. Another feature of the developed prototype concerns the possibility of communication via voice messages between the person with AD and his/her member. Additionally, our prototype integrates Google assistant for supporting the persons with AD and therefore answering his/her questions, reducing social isolation and predicting his/her psychological status. So far, our prototype tracks the location of the person with AD and sends an alert if the person leaves a specified area. Our prototype is useful for persons who are affected by mild and moderate AD. It supports them for remembering their family members and recognizing other people after decrypting the extra information hidden in the images. Our results show that our prototype is effective for detecting the images of the family members of a person with AD while ensuring a high accuracy and precision compared to other benchmark techniques.


I. INTRODUCTION
Among the major public health challenges, detection of AD is very important because it affects a very large number of people. As reported in [1], around three million people have been affected as patients or family members by Alzheimer in France in 2020. The percentage of individuals that have been affected by AD is 1%, 20% and 40% for those at the age of 65 to 69, 85 to 89 and 90 to 95 respectively [2]. AD is known as a degenerative disease that causes a progressive decline in cognitive ability and memory. There is currently no cure for AD. However, Artificial Intelligence (AI) has enabled a shift The associate editor coordinating the review of this manuscript and approving it for publication was Hiram Ponce . to more preventative medicine, allowing the prediction and detection of many diseases and therefore saving time in treatment. Besides that, AI tools have shown their efficiency for designing assistance based solutions including mobile applications, devices and chatbots that provide support to patients. Moreover, deep learning is a field of machine learning which adopts a structure that is similar to artificial neural networks (ANN). It embraces a set of algorithms that run on a multilayer architecture to analyze data and draw conclusions. The ANNs used by deep learning do not require feature extraction. The layers have the capacity for learning implicitly the representation of raw data [3]. According to the literature review on AD, deep learning algorithms have been investigated for the detection of AD. Notably, the performance of CNNs for detecting AD exceeds the performance of existing machine learning algorithms. It is worth to be mentioned that some AD assistance based solutions exist. The majority of them support IoT applications but they did not integrate security features and did not investigate deep learning. Thus, our article develops a solution based on facial recognition and security tools to help Alzheimer's patients and improve their lifestyle. In this context, using our prototype, the person with AD will have the ability to identify family members. More importantly, we have designed a CNN for facial recognition. Additionally, the messages exchanged between the patient and his/her family member are encrypted using steganography. Two other features provided by our solution concern sending notification when the patient with AD will leave a safety zone and allowing voice communication.
Compared to the state of art, our contributions are summarized as follows: • Designing a collar that is easy to wear by a person with AD.
• Designing a CNN that extracts the points of interests of the mouth, nose and eyes for binary classification of images into family member/not family member.
• Hiding the identity of the person who is not registered in the database. Steganography is investigated to hide the identity as well as extra information about the relationship between this person and others. The aim is twofold: protecting the transferred identity and supporting the person with AD to recognize more people during future conversations.
• Enabling voice conversations with a person with AD through Google assistant. Statistics can be extracted from the conversations carried out to identify psychological status of the person with AD.
• Tracking the person with AD and sending an alert in case he/she leaves the safety zone.
• Securing the private information of the person with AD through Secure/Multipurpose Internet Mail Extensions (S/MIME) protocol. Thus, section 2 of this paper discusses related work. Then, section 3 explains our systems, while section 4 details our results. Finally, section 5 concludes this paper and highlights potential future works.

II. RELATED WORK
This section briefly describes the works that are related to our paper. We mention that the literature includes two research categories. The first category has focused on detecting AD, whereas the second one has focused on the assistance of persons with AD.
Donghuan Lu et al. [4] have designed a new multi-scale and multi-modal deep neural network (MMDNN) to support the early examination of AD. Hence, the proposed MMDNN enables combining information from magnetic resonance imaging (MRI) and fuorodeoxyglucose positron emission tomography (FDG-PET). More importantly, the developed MMDNN utilizes MRI images for analyzing the regions of the brain, while FDG-PET images are very essential for analyzing the activity of the tissues. Further, MMDNN extracts features by segmenting the images and has the ability to conserve the metabolism and structural information. The authors of [4] have focused on comparing the accuracy achieved by the designed MMDNN to the accuracy of other techniques. Thus, the accuracy of MMDNN is 86.4%. It outperforms the accuracy of the other benchmark techniques.
Researchers in [5] have proposed a set of densely connected 3D convolutional networks (3D-DenseNets) for AD diagnosis and mild cognitive impairment (MCI) using a probability-based fusion approach. Moreover, 3D-DenseNets is based on MRI images and it facilitates the gradient propagation by linking each layer to all its subsequent layers. Additionally, DenseNets utilizes a softmax function for calculating the output. The authors of [5] have investigates ADNI dataset for testing DenseNets that achieved a classification accuracy of 97.52%.
In the study presented in [6], the suggested method is based on the estimation of the three-dimensional (3D) displacement field for extracting the relevant regions from MRI images of normal elder participants. Two variants of support vector machines (SVM) including Generalized eigenvalue proximal SVM and twin SVM have been explored. The results illustrated in [6] show that the designed method attains an accuracy of 93.05%, a sensitivity of 92.57% and a specificity of 93.18%.
Morteza Rohanian et al. [7] presented two fusionbased multimodal deep learning models that simultaneously use transcribed voice and acoustic for extracting features and detecting AD from a set of speech recordings. The results show an accuracy of 84% and a root mean square error (RSME) of 4.26.
The contribution introduced in [8] has presented a CNNbased framework for AD classification. The proposed CNN includes five layers and it investigates ADAM optimizer for detecting AD from MRI images. It achieved a classification accuracy of 97.5% on the Alzheimer's Disease Neuroimaging Initiative (ADNI) data set for the binary classification of AD and cognitive normal.
The work detailed in [9] focuses on detecting the presence of the AD at the early stage using three components including Hippocampus, Corpus Callosum and Cortex. Thus, the classification method is based on SVM and the proposed system yields 90.66% for the accuracy of early diagnosis of AD.
The idea presented in [10] is based on a simple CNN architecture that detects AD from 2D and 3D structural brain scans. VGG19 has also been exploited in [10] for applying the transfer learning principle. So, the results obtained in [10] are very significant as the achieved accuracy is respectively 93.61% and 95.17% for 2D and 3D classifications respectively; while the VGG19 model has attained an accuracy of 97%.
The work presented in [11] has performed multimodal data fusion based on MRI data to understand AD. Each developed CNN has been trained for each data modality. Then, integrated classification has been applied using decision trees, random forests, SVMs, and k-nearest neighbors. The results discussed in [11] demonstrate that the integration of multiple modalities improves the prediction accuracy. The main limitation of the work presented in [11] is that the size of the dataset is limited.
The authors of [12] have discussed the potential of AI techniques that are suitable for big data analysis for AD diagnostic. However, the integration of the appropriate modalities is recommended to improve the performance of the prediction of AD [12].
Another contribution has been introduced in [13] based on the analysis of available models for predicting AD. The proposed model [13] includes five steps. The pre-processing, selection of the attributes and the classification using the association rules are the main steps of the suggested model that has been described in [13].
Authors of [14] have developed a deep learning model for predicting AD from fluorine 18 ( 18 F) fluorodeoxyglucose (FDG) images of the brain. Thus, the architecture of the model proposed in [14] is based on CNN of InceptionV3. The developed model in [14] has been first pre-trained and then data augmentation has been performed. The results obtained in [14] demonstrate that the proposed approach achieves an area under ROC of 98%.
Moreover, the work detailed in [15] has focused on the fusion of multi-modal-information MRI and PET for AD prediction. The model proposed in [15] is based deep on Boltzman machine to find the hierarchical feature representation. The results illustrated in [15] demonstrate that the proposed approach outperforms other techniques in terms of the accuracy.
Let-Net5 has been investigated in [16] for binary classification of MRI data. In addition, CNN has been used and the accuracy of tested data has achieved 96.85%. We mention that Let-Net5 has been tested for a small sample of data.
Different from the research works that deal with low-level features, the authors of [17] have used a stacked auto-Encoder to find hidden feature representation from MRI and PET. The accuracy of the method presented in [17] has attained 95.9%. The principle limitation of the deep learning model proposed in [17] is that it does not integrate different modalities.
The system developed in [18] consists of a wearable electronic device that is equipped by sensors for supporting patients with AD and improving their life style. Furthermore, the electronic device enables locating the patient on the map and reminding him/her for medication times. It provides also a button for requesting assistance in case of emergency. The persons who participated in the evaluation have reported that the designed wearable device is easy to use. Additionally, the system designed in [19] provides voice based assistance for patients with AD by answering their questions regarding healthy food. We mention that the main limitation of the work presented in [19] is that the performed tests have not involved persons with AD.
The system designed in [20] has aimed to track vital signs and the position of the person with AD through a set of sensors and the Global Positioning System (GPS). The collected data are sent every five minutes to the caregiver trough SMS for analyzing the health status of the person with AD.
An interesting mobile application has been proposed in [21] to support patients with mild and moderate AD. The application is based on facial recognition and GPS tracking for enhancing the daily communication of patients with AD. Further, the face recognition process is based on machine learning. It supports the patient with AD to remember his/her family members. Additionally, the mobile application enables sending notification to remind the patients about their daily tasks. The main feature of the work presented in [22] is that it provides a framework for predicting AD and assisting patients with AD. The prediction is performed using a recurrent neural network (RNN) that investigates movement's tracking data from Daphnet dataset. Then, a combination of CNN and timestamp window based natural language processing scheme is employed for abnormality tracking. More precisely, the developed CNN aims at detecting the emotion of person with AD. Further, the proposed framework provides assistance regarding the daily activities including meal, bath, medicine and hydration activities.
According to the above state-of-art regarding the solutions for AD, some comments need to be highlighted: • The works presented in [4]- [17] have targeted the detection of AD. In this perspective, the research papers that are based on deep learning and precisely CNNs have demonstrated outperforming results for detecting mild and moderate AD from MRI images.
• The works presented in [18]- [22] have targeted the assistance of the persons with AD. More importantly, table 1 compares AD assistance based systems. It is clear from table 1 that the work presented in [22] provides many features compared to the systems designed in [18]- [21]. However, the authors of [22] did not provide solutions for face recognition and voice assistance for a person with AD. Another main comment concerns the lack of security in all related works, while IoT based applications require security mechanisms for protecting data. So far, the work presented in [21] is based on machine learning for face detection. Whereas, the research on deep learning has demonstrated its efficiency compared to machine learning techniques. Different from the above related works, our system investigates CNN for face recognition. It allows supporting patients with AD for classifying the person to whom he/she is talking. Additionally, our system supports voice communication through Google assistant and tracking via GPS system. Thus, a notification is sent if the person with AD leaves his/her safety area. The security feature has been integrated to our solution through hiding the identity and extra information into the images that are not stored in the database.

III. PROPOSED SYSTEM
According to the US National Library of Medicine [23], a normal human body temperature ranges between 97 • F (36.1 • C) and 99 • F (37.2 • C). Thus, the camera can detect the temperature of objects with an accuracy of ±0.2 • C. However, if the temperature is more than 99 • F (37.2 • C), our camera cannot not detect the person's face.
Indeed, we suggest to developing a prototype in the form of a collar worn by the person with AD so that he/she would be happy to wear it by doing facial recognition (RF). Therefore, we use an intelligent algorithm that will classify the detected images into two categories based on a provided database (known as: family member, unknown: is not a family member). Then, a family member can access the database and add a particular face. In addition, we include tools to ensure the physical safety of the patient with a AD and the confidentiality of his/her personal data against attacks. As a result, a system for tracking the current location of Alzheimer's patient is defined using smart alert notifications, which are activated in case of leaving a specific area. As a result, the proposed prototype provides psychological support and mobility support services for persons with AD. Moreover, the progression of AD varies a lot from one person to another one. More specifically, our prototype allows assisting persons with mild and moderate AD. The goals of the designed collar that is supported by an IoT application is to assist a person with AD to remember his/her family members, recognizing more people through the decrypted identity and extra information hidden in the images that are not stored in the database. The other requirements of our prototype concern tracking, safety, and voice assistance. The overview of our proposed solution is shown in Figures 1, 2.
In the following, we present the different components of our system. Figure 1 illustrates that our prototype enables both face recognition and voice detection. The face recognition is based on a CNN. However, if the provided image did not exist in the database, steganography will be utilized to hide 44308 VOLUME 10, 2022 the identity and extra information about the person. Further, our prototype allows tracking the person with AD and may send an e-mail notification. Figure 2 shows that our prototype performs a pre-processing including level normalization and median filter. Then, the designed CNN extracts the points of interest of the eyes, nose and mouth for classifying the input image as family member/not family member.

A. COMPONENTS OF FACE RECOGNITION 1) PRE-PROCESSING
The role of this step is to eliminate the interference caused by the quality of the optical or electronic devices during the acquisition of the input image, in order to keep only the essential information and therefore prepare the image for the next step.

2) DEEP FEATURE EXTRACTION
information is extracted from the image and stored in memory for further utilization in the decision phase. The extraction of features of the face is done through the detection of points of interest of the mouth, nose and eyes. The efficiency of this step has a major impact on the performance of the face recognition system.

3) COMPARISON OF CHARACTERISTICS (CLASSIFICATION AND DECISION)
It consists of modelling the parameters extracted from a face or a set of faces of a person based on their common characteristics. Several approaches are found in the literature, the simplest of method consist to apply the distance calculation (similarity search).

4) DECISION
This is the step that makes the difference between a person's identification system and a verification system. Identification system consists of finding the model that best corresponds to the input face. It is characterized by its recognition rate. However, a verification system has the matter of deciding whether the input face is that of the proclaimed individual or it is an impostor. More precisely, our prototype id dedicated for identification. We mention that the CNN architecture of our system is made of one input layer, multi types of hidden layers and one output layer [24]. The layers provide two distinct functionalities. The first part of a CNN works as a feature extractor from the images. An image is passed through a succession of filters, or convolution nuclei, creating new images called convolution maps. Some intermediate filters reduce the image resolution by a local maximum operation. At the end, the convolution maps are flattened and concatenated into a vector of features, called the CNN code. The first kind of hidden layers is responsible for convolution and the other one is responsible for local averaging, sub sampling and resolution reduction. The third hidden layers act as a traditional multilayer as shown in Figure 3. With reference to our architecture that has been illustrated in Figure 1, CNN is integrated to the webcam for enabling the edge computing concept. The latter improves the efficiency and reduces extremely the delay.

Solution1: Steganography
This is the technique of hiding a confidential message in digital files [25], for example photos in our case. The image chosen for this purpose is called the cover image and the image obtained after steganography is called the stego image. In our proposed solution, in order to create an Alzheimer's support system, a family member can hide the data concerning the identity of the unidentified person and only the recipient can decode it using a key.
Solution 2: allows to send voice mail message to Gmail from someone with AD in case he/she is unable to read. Transcripts of the forwarded voice mail messages will be displayed in the e-mail application of the person with AD. A notification service is also available to alert the person about the reception of an e-mail. Our application allows to VOLUME 10, 2022 choose voice memos and then record the voice mail message to inform the recipient about the identity of the unidentified person.

B. COMMUNICATION SUPPORT SYSTEM
The idea is very simple and consists to enable the person to speak a word for capturing the voice that will be recorded in the form of acoustic vectors. The series of acoustic vectors completely characterizes the evolution of the spectral envelope of the recorded signal. The next step includes the analysis of the unknown signal as a series of similar acoustic vectors, and Google assistant will respond to any issue of the speaker who is talking via a kit. The same word can in fact be pronounced several times since the person who has AD has difficulties in remembering as shown in figure 4.
It is important to note that assistance based technology has emerged and it has a great potential to improve the quality of life of elderly people. It has improved communication with family and caregivers; this is also convenient for social care providers and the elderly can communicate easily with simple sentences. It has also offered more autonomy for people with mobility issues and reduced vision impairments. If the person has trouble getting around or cannot remember the location, he/she can ask Google Assistant to remember places that he/she has visited previously.

C. SAFETY MONITORING AND SECURITY
Our system aims to ensure the physical safety of the patient by monitoring him/her remotely using a portable device for elderly patients with AD and a given listening device. The network is exposed to several types of attacks that may concern the data generated by sensors or the network. To do this, we offer a comprehensive system that ensures the physical safety of the patient, as well as the security of the his/her personal information. Our location monitoring system protects the patients with AD while preserving their personal freedom. We used the GPS to properly locate the person with Alzheimer's as shown in figure 5. Using a cellular data service plan, GPS updates the location every 2 minutes and plots the location data on a map. Designated geo-zones can be established and if the patient leaves the zone, an alert is sent by e-mail. The designed collar records the movements of patients throughout the day. Later, in case of exit from the security zone, an e-mail will be sent to the family member indicating this information. The messages that will be sent are very sensitive, because they contain private personal information. It is therefore important to protect them. Gmail allows to send messages in a confidential manner to protect sensitive information from unauthorized access. Our prototype offers the possibility for integrity, authentication and non-repudiation using digital signatures of an e-mail by enabling S/MIME protocol. This protocol is a widely accepted protocol for sending digitally signed and encrypted messages and which also helps to improve data privacy and security (using encryption).

IV. IMPLEMENTATION AND RESULTS
Our prototype has been tested by 162 volunteers. The type of development board used is Raspberry pi3 provided by 4 GB of RAM. It is a single board computer, powerful enough due to its small size. In particular, our board is considered as a smart machine [8] that is used for radio frequency transmission, GPS and Google Assistant. Additionally, our implementation is based on the following hardware components: • The Xtrike me Microphone has a stunning sound quality that records the voice perfectly without background noise.
• A buzzer that is a kind of loudspeaker with low power, which will emit ringing using repetitive beeps. • An earpiece that is a device which is placed in an ear and which allows sound content to be reproduced. It is used to listen to the answers of the Google assistant. We have used the NEO 6MV2 GPS module with different microcontrollers. When supplied with a direct voltage of 3.3 V, it receives the signals emitted by geolocalization satellites and continuously transmits information by universal asynchronous receiver-transmitter (UART) communication. The hardware and software components are shown in figure 6. For facial recognition on the Raspberry Pi3, the entry is an image from a webcam. First, OpenCV, which is a library of open source computer vision and machine learning software is executed for determining whether there is one or more faces in the image. If the faces are found, we identify their coordinates and conditionally cut out only that part of the general frame, transfer it for recognition and match it with the available images in the database. We have installed the OpenCV Contrib version which allows us to also develop facial recognition and object recognition systems. Mainly, our python implementation is based on dlib library that is appropriate for solutions based on machine learning and deep learning.

A. FACIAL RECOGNITION EVALUATION RESULTS
The core idea of our system is based on the use of the investigation of CNN algorithm. Factors influencing the identification include variations in pose, as wide-angle cameras can provide good detail about nearby objects. However, the details are much worse on closer objects. In addition to variations in lighting that disturb the face recognition process as shown in the figure 7. According to our test results, our prototype enables a good identification even that the person is wearing a mask. Factors that influence the identification performance of people include the variation in facial expressions such as images and facial elements including the mouth and eyes.
Thus, the main result is the confusion matrix, which allows us to understand if there is a misclassification. Table 2 presents the performance results of the proposed algorithm. It is clear that the achieved accuracy is very high and the execution time is low. VOLUME 10, 2022

B. GOOGLE ASSISTANT
Google will search for the questions and enables conversations to reduce social isolation and depression. If the person asks the question ''Do you love me'', our prototype performs statistics to know if the question is repeated several times and therefore classifies it as a message. The results for google assistants are shown in Figure 9. To pay more attention to the person talking to the assistant, our prototype allows access to information from the calendars and other sources (weather, position. . .) and there is no problem if the person forgets the location of last week or has difficulty for concentrating with Google as he/she can repeat the questions at any time.

C. GPS
The messages that will be sent to a family member are secured by the protocols used by Gmail. More specifically, an alert message that contains the word and the location which will be sent every two minutes in case that the wearer of our prototype remains outside the safety zone and a buzzer gives alerts to the bearer of our prototype. If the person forgets the location of the last week or have concentration difficulties, he/she can go back to the history to revisit the places that he/she has visited. Additionally, a family member can make statistics on the cognitive state of his father / mother who has Alzheimer's since he has symptoms similar to that of Alzheimer's. The test results for the GPS system are shown in figure 10.

V. CONCLUSION AND FUTURE WORK
Regardless of the existence of many solutions for AD, the number of AD assistance based systems is limited. Furthermore, facial recognition and security features are still a challenging problem for current systems that help improving the lifestyle of persons with AD. This paper has proposed to design a simple to provide assistance to persons with AD. Indeed, we have presented a facial recognition prototype that is based on CNN. In addition, steganography encryption has been integrated to protect the identity of the person who is not registered in the database and hence, supporting the person with AD to recognize more people during future conversations. So far, our prototype includes a GPS system that is capable of providing protection by specifying the location of the patient. Thus, a voice alert will be sent to a family member in case the patient leaves the predefined space zone. To help persons with AD, the proposed system includes a psychological monitoring system through Google Assistant. As shown in the results section, our prototype achieves significant results in terms of the accuracy and precision compared to the benchmark techniques.
As future work, as a continuation of the research that we have carried out, we propose the recognition of persons by a thermal camera that identifies an object among other objects. We also envision performing 3D facial reconstructions to solve the pose problems in the identification of a person and integrate the voice to ensure a reading of the identity if visual problems exist. Facial reconstruction has many applications and the use of information theory to increase the efficiency and speed of the system is another direction that can be explored.
RANIA CHOKRI received the master's degree in network and telecommunications engineering from the National School of Electronics and Telecommunications of Sfax. Her research interests include Alzheimer detection, deep learning, and security. VOLUME 10, 2022 WASMA HANINI received the National Diploma degree in industrial computer engineering from the National School of Electronics and Telecommunications of Sfax, in 2015, and the Ph.D. degree in electrical engineering from the National Engineering School in Sfax, in 2020. His research interests include electrical systems, artificial intelligence, electronics, and the IoT. SAMIA ALLAOUA CHELLOUG received the Engineering degree in computer science, the master's degree in computer science, and the Ph.D. degree in networking from the University of Constantine, Algeria, in 2003, 2006, and 2013, respectively. From 2006 to 2013, she was an Assistant Professor with the Faculty of Computer Science, Constantine University. In August 2013, she joined the Department of Networks and Communication Systems, Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia, where she is currently an Assistant Professor. She has published more than 15 papers in journals and conference proceedings and she has reviewed some ICCC15 conference papers. Her current research interests include wireless sensor networks, body area networks, cloud computing, cognitive radio, mobile wireless networks, vehicular networks, the Internet of Things, and pervasive computing.
AMEL MEDDEB MAKHLOUF received the Ph.D. degree from SUP'COM, Tunisia, in 2010, and the Habilitation degree, in December 2020. From 2001 to 2004, she worked as the Chief of the Certification Unit, NDCA. Since September 2010, she has been working as an Assistant Professor at ENET'COM, Sfax, Tunisia. Since 2020, she has been the Head of the Telecommunication Department. She was a supervisor of more than 30 master's projects and 14 Ph.D. She coauthored more than 40 papers, published in international journals and refereed conferences. Her research interests include security of vehicular networks, cloud networks, and BSN.