Activity Detection for the Wellbeing of Dogs Using Wearable Sensors Based on Deep Learning

Among pets, dogs are very famous in the whole world. The owners of dogs are very cautious about the well-being of their dogs. The well-being of dogs can be ensured by continuous monitoring of their activities. Studies related to activity detection have gained much popularity due to the significant progress in sensor technology during the last few years. Automatic monitoring of pet applications includes real-time monitoring systems and surveillance which detect the pets with high accuracy using the latest pet activity classification techniques. The revolution in the domain of technology has allowed us to obtain better results using latest techniques. Convolutional neural networks (CNNs) 1D recently become a cutting-edge approach for signal processing-based systems such as patient-individual ECG categorization, sensor-based health monitoring systems, and anomaly identification in manufacturing areas. Adaptive and compact 1D models have several advantages over their conventional 2D counterparts. A limited dataset is sufficient to train a 1D CNN efficiently while 2D CNNs require a plethora of data for training. Its architecture is not very complicated, so it is suitable for real-time detection of activities. The main goal of this study is to develop a state-of-the-art system that can detect and classify the activities based on sensors’ data (accelerometer, and gyroscope. We proposed a 1D CNN-based system for pet activity detection. The objective of this study was to recognize ten pet activities such as walking, sitting, down, staying, eating, sideway, jumping, running, shaking, and nose work respectively, using wearable sensor devices based on deep learning technique. The data collection procedure for this study was conducted with 10 dogs of different breeds, sex (male=7, female = 3), age (age = 4±3), and sizes (small, medium, large) in a healthy environment. After collecting the data, the following steps, namely data synchronization, and data preprocessing were considered to remove the irrelevant data from the dataset. To overcome imbalanced problems in the dataset we used the class-weight technique. Subsequently, we applied 1D CNN algorithm using the class-weight technique. The model with the class-weight technique showed 99.70% training accuracy and 96.85% validation accuracy. The 1D CNN approach will be helpful for real-time monitoring of activities and for tracing the behavior of dogs.


I. INTRODUCTION
Pet activity detection (PAD) is a dynamic and arduous research topic. Activity detection systems are a broad field The associate editor coordinating the review of this manuscript and approving it for publication was Agustin Leobardo Herrera-May . of research and evolution, with a focus on state-of-the-art machine-learning techniques and, a revolution in the domain of hardware architecture. Research curiosity in automatic techniques provides an incessant assessment of the health and well-being of pets' activity recognition, and the welfare of pets has recently been growing. Pet activity recognition VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ and classification are imperative for a vast range of applications, including monitoring pets and keeping track of pet activities. The rapid development of technologies, especially sensor devices, and the least cost of various sensors, make them easily accessible for the detection of daily activities of pets. Wearable devices are used for activity recognition. Inertial sensors and embedded systems seamlessly enable wearable devices in the activity-recognition process. Nowadays it has become an innate part of daily life and is extensively applied to innumerable common realms including welfare assessment of animals, medical monitoring, rehabilitation activities, health management, remote control, and action recognition [1]- [11]. In addition, an accelerometer has been used in humans for gait examination [12]- [15] as well as for the analysis of circadian rhythms [16], [17], and sensors have been used to observe and monitor lameness in horses for the past few years [18]- [21]. An accelerometer has been used to monitor various activities in dogs [22]- [26], types of activity [27], cognitive issues, and lameness recognition [28]- [30]. Wearable sensors combining embedded systems with acceleration and gyro sensors have been established for activity recognition and are used in everyday life and sports activities. The benefits of acceleration and gyro sensors combined with embedded systems in wearable instruments for motion monitoring and recognition are that no exterior environment sensors such as cameras, infrared sensors, or radars are required for these wearable instruments [31]- [33]. Additionally, their diminutive size, low cost, lightness, reduced power consumption, acceleration, and gyro sensors in wearable devices provide a solution for recognizing sports activity. According to Khalifa et al., KEH (kinetic energy harvesting) may help overcome the battery problems in wearable devices. KEH is chiefly used as a generator and human activity recognition sensor, which reduces the power consumption of the sensor. The results show that activity detection by KEH can overcome system power consumption by 79% [7]. The most imperative step in understanding the behavior of an animal, including the activity patterns, is to create certain ethograms associated with that species by monitoring the physical movements and body postures [34]- [36]. There are two main approaches for detecting the activities of pets. Continuous monitoring by humans or using some wearable devices. The first method is not viable because it may affect the activity of the pet and the natural behavior may be disturbed and cannot be properly analyzed. The use of sensor devices, such as gyroscopes and accelerometers, is another effective approach for data collection from pets without interfering with their normal activity. The automated technique that is currently available applies sensors and can be directly tied to dogs, or it creates an environment while collecting the data with the help of sensors [37]. During the past few years, it has been noticed that automated methodologies for the discovery of behaviors are getting well-liked because the tied sensors in the automated methodology have the potential to differentiate numerous activity patterns. In the realm of biosignal processing, it is a prominent fact that comprehending data from the various sensors fixed to one's body allows boosting the level of accuracy. To make decisions regarding one's health, activity, and state of mind, there are many factors to contemplate, and therefore, leveraging various sensors in this situation is deemed necessary.
The main objective of this study was to analyze the activities of pets based on state-of-the-art approaches to their well-being. Although researchers have proposed various techniques, those techniques have several drawbacks such as some of them have used only accelerometer data and some of them have used raw data and did not perform feature engineering techniques. In this research study we considered these factors and we have used two sensors, that is, an accelerometer sensor, a gyroscope sensor on the neck, and two sensors on the tail, that is, accelerometer and gyroscope sensors and we performed feature engineering and applied the 1D CNN model.
• Applying state-of-the-art 1D CNN deep learning technique on pet activity sensors' data.
• Extracting different features from the raw signal data.
• We are among the pioneers who applied the 1D CNN technique to pet activity sensor data for the detection of dog activities.
• Ten activities have been classified using 1D CNN.
• To address the imbalance problem, we have used class weight approaches.
• The class-weight approach proved to be suitable for the detection of pet activities The remaining sections of this paper are organized as follows. Section II shows a brief overview of related work. Section III describes the materials and method. Section IV explains the activity detection algorithm. Section V highlights the complete workflow. Section VI is related to experimental results and discussion. Section VII concludes our research study.

II. RELATED WORK
Nowadays activity detection is getting attention and has been growing rapidly, which is an effective way to ensure the wellbeing of animals.
Ladha et al. proposed a KNN machine learning model for the classification of 17 different activities of pets. The data was collected using an accelerometer from 13 different bread, weight, ages, and both sexes. For the ground truth, the dog's activities were filmed using a camera. The annotation procedure was performed by the expert against filmed video footage. The fact in this study is that for data collection procedure has not been conducted in a proper environment. The KNN model performance showed 68.6% accuracy for overall activities and observed that due to erroneous annotation issues some activities results not well. They think that this is the first robust model for the activity's detection in naturalistic environments [38]. S Aich et al. represented a method that could be used to make an automatic system for pet activity and emotion detection. They investigated different queries like what types of data should be used and the location of the sensors. They applied different machine learning algorithms such as random forest, KNN, SVM, naive Bayes, and ANN for activity and emotion detection. Among those classifiers, ANN performed well for activity and emotion detection. They found 96.58 percent accuracy for activity and 92.87 % accuracy for emotion detection [39]. Gerencser et al. proposed a support vector machine model (SVM) for activity detection. Accelerometer and gyroscope sensors were used for the collection of data. To analyze the pet activities with better performance, the feature extraction method was used, and different 126 features were extracted from the data such as standard deviation, average, higher moments, extrema values, vector lengths, etc. The extracted feature was fed to the SVM model for training after the training evaluation method was performed to check the performance of the model and used a different combination of data sizes to check the robustness. The model showed overall 91.3% accuracy, with seven activities [27]. Prevents serious issue and disclose the disease in the early stage and motivate the owner of an animal to look up the early veterinary recommendation. Uijl et al.
proposed an accelerometer data-based model that can detect particular changes in activities or behaviors. The data was collected from 51 healthy dogs of different ages, weights, and breeds. The overall results mention as 95% classified correctly in walking, and trot, eat, drink, canter, headshake above 90% [40]. Rahman et al. represented a machine learning approach, placed the accelerometer at different locations, and compared the results. Different statistical features were computed from sensor data for the sake of classification analysis. The purpose of this study was to understand how different behaviors were effectively classified using the computed statistical feature from the sensors attached to different positions on the animal's head. They found that the location of the sensor device at the halter gave better experiment results as compared to ear tag, and collar data [41]. The monitoring systems using the latest technologies are gaining popularity [42]. Yashari et al. purpose a novel study, monitoring dogs' activity based on a smartphone accelerometer (Whistle). The purpose of this study was to assess this novel accelerometer. Although the entire activity time given by the Whistle-based technique offers a low-cost procedure for getting real-time activity data from dogs at home. But there are some limitations in this study, main issue is battery life, which requires manual set derivation and intensity of the activity, Wi-Fi, and Bluetooth to transmit the data [23]. De Seabra et al. proposed a method to analyze the pet activity using accelerometer and gyroscope data. The sensors were mounted on the back of dogs. The goal of the proposed study was to assess the pets' well-being and health state and to develop a device that could track the activity of the pet, behavior, and physiological markers of the pets. This was a preliminary work to analyze the feasibility of the sensor devices for pet activity detection [43]. Decandia. M et al. represented a machine learning-based system to monitor the different activities. Behavior monitoring of grazing animals is crucial for the control of the grazing system. An accelerometer was used to analyze the activity, and the main objective of the study was to discriminate various behavioral activities. The fifteen different features like mean, variance, standard deviation, etc. were extracted from the raw signal data for each axis and also found the resultant vector using the feature engineering technique. The system showed better results to distinguish the different activities and they got 89.7% accuracy in terms of classification [44]. P Chakravarty et al. developed a hybrid framework that comprises biomechanical variables and a classification mechanism that took place on each node to find the different behaviors based on threshold values of features. Accelerometer and GPS were used for the collection of data to detect the vulture's behavioral modes. A support vector machine with a linear kernel showed an overall 95 % classification accuracy result for the individual scenario. Meanwhile, their proposed approach showed 2.7 percent better performance as compared to other approaches [45]. S Venkatraman et al. Proposed a method to recognize the activity pattern and neural behavior using accelerometer data in rats. The designed sensor was very tiny and lightweight which is reliable to use for small animals like rats. A neural network approach was used to detect the different behavioral activities pattern. Grooming, eating, and standing only three activities were detected using that method [46]. S Grunewalder et al. Proposed new machine learning-based techniques. The objective of this technique was to analyze the continuous data from the data storage devices and at the same time behavior detection. This data combination allows biologists to examine the behavior of Cheetahs at an unattainable degree of detail and precision; nevertheless, continually recorded data are useless unless the large amount of raw data generated can be consistently converted into actual behavior. To solve this challenge, they combined an SVM (support vector machine) and a hidden Markov algorithm to characterize an animal's behavior. The technique was deployed on six cheetahs. They were able to classify every 5 mints activity score into a sequence of three fundamental behaviors such as feeding, mobile, and stationary. The accuracy of their classification model was determined via cross-validation, however, the accuracy for different classes decreased as the size of the sample of direct observations reduced. Their model has shown validation accuracy between 83 percent-and 94 percent [47]. SM et al. Proposed a 1D CNN-based technique for human activity recognition. The data was collected using a smartphone accelerometer. Three activity data were collected such as walking, staying, and running respectively. The collected data of the three-axis were transformed according to a data format that can be fed to the 1D CNN model for training. The performance for ternary activities in the model has shown 92.71 percent accuracy, and the other baseline algorithms such as random forest showed 89.10 percent accuracy [48]. CT Yen et al. Proposed a technique based on the wearable device that was able to detect the daily six human activities such as walking, walking downstairs, walking upstairs, standing, sitting, and lying using a deep learning model. The device was fastened on the waist of the human body. Accelerometer and gyroscope sensors were used for the collection of data. They applied 1D CNN-based algorithm. They have used two different datasets in their research study i.e. University of California (UCI) dataset and the second one was their recorded dataset. They found that training accuracy using the UCI dataset was 98.93% and their recorded dataset training accuracy was 97.19% respectively. The testing accuracy was 95.99% and 93.77% [49].
Jinah Kim et al. [50] presented multimodal data-based dog behavior recognition. They used both the sensor data and camera data and fused them for this purpose. Object detection techniques like FasterRCNN, YOLOv3, and YOLOv4 were used. The recognition accuracy of YOLOv4 was highest compared to the rest of the models. They also checked the performance with single data-based and multimodal data-based models. The multimodal data-based model i.e CNN-LSTM showed the best performance. Huasang Wang et al. [51] developed a behavior monitoring system for dogs. The system was able to detect psychological disorders like separation anxiety (SA) in dogs. They used Stacked Long Short-Term Memory (LSTM) and fuzzy logic for the development of this behavior monitory system. Eight dogs were included in this research study and data was collected from the wearable sensor device. The system achieved an F1-score of 0.86.

III. METHODS AND MATERIALS
This section describes the methods and materials we used in our study.

A. DATA COLLECTION AND IMPLEMENTATION ENVIRONMENT
The data was collected from 10 different dogs of different genders, breeds, ages, and sizes. The data was taken with the consent of the dogs' owners. Wearable sensor devices were used to collect the data and the sensors were placed on the neck and tail of the dogs. The sampling frequency of 33.33 Hz was used to investigate the activity of the dogs. The sensor devices are incorporated with two types of sensors i.e accelerometer and gyroscope. These two types of sensors enabled the wearable device to measure the rotational and linear motions of the dogs in all directions. These wearable devices are lightweight and can easily be placed on the neck and tail of the dogs without causing discomfort to the pets during their movement. The neck worm device is 16 g in weight and has a dimension of 52 x 38 x 20.5 mm. Likewise, the weight of the tail-worn device is 13g and it has the dimension of 35 × 24 × 15 mm. The accelerometer has a scale factor of −4 to 4 g while the gyroscope has −2000 DPS to +2000 DPS. Sweet Solution, Busan, South Korea has manufactured these sensors. The data has been collected from trained dogs under the supervision of trainers who were responsible for determining the activity of dogs while using video recordings and IMU data recordings. The trainer checked the position of the sensors ensuring their proper placement on the neck and the tail of the dogs frequently. In order to extract the data of specific activity, the trainer instructed the dogs to perform that particular activity. The dogs performed the specific activity accordingly as they were instructed. At the same time, the IMU data was recorded by one person and one of the other persons records the activity using a video recorder. The video was recorded in line with the sampling rate of the sensor devices i.e same number of frames was recorded per second as the sampling rate of the device. In other words, 33 frames were recorded in one second as the sampling rate of the wearable device was 33 samples.

B. DATA PREPARATION
Data preparation and data cleaning play very important roles in obtaining the optimal performance of any artificial intelligence-based model, therefore we performed different data preparation techniques to make the data fit to be used for model development. The data extracted from the devices were noisy and irrelevant and noisy data were removed from the dataset. We applied a 6 th -order Butterworth filter having a cutoff frequency of 3.667 Hz to remove the noise and inconsistencies from our dataset. This order of filter blocks maximum noise therefore we set this order. Likewise, the cutoff frequency was chosen based on the exploratory data analysis. This technique also helps to filter out the sensor data which are affected by gravity and makes the data smoother and less dependent while reducing the influence of abrupt changes on the accelerometer data.

C. STATISTICAL FEATURE ENGINEERING
In order to obtain the important statistical features, we applied feature engineering to the sensor data. Several features were derived from the accelerometer and gyroscope sensor data. Feature engineering enables the extraction of the most relevant and important features from the pool of data. For the feature engineering, ten features for each axis were performed on the accelerometer and gyroscope. The features were derived by considering a certain number of samples. The derived features were, standard deviation, mean absolute deviation, mean, minimum, maximum, interquartile range, energy measure, skewness, and kurtosis.

D. CLASS WEIGHT TECHNIQUE
Class weight is one of the approaches used for balancing the data [52]. In this technique, we take care of the minority samples more while training the model, and to calculate the loss function a weighting mechanism is developed. Different weights are assigned to majority and minority classes according to the imbalance scenario in the dataset. In order to keep a balance among the classes, a threshold should be defined so that class weights can be increased or decreased. This will help in preventing the biasing of the algorithm towards any specific class. The formula for class weight can be defined as where wi represents the weight of each class and i represents the class. The n_instances denote the total number of instances or rows in our dataset whereas n_classes represent the overall unique classes in the class label. The total number of rows in each class is denoted as n_instancesi.
The weighting mechanism adopted in this study is listed in Table 1 below.

IV. ACTIVITY DETECTION ALGORITHM
A pet activity detection algorithm was developed which included the collection of biosignals from wearable devices i.e., accelerometer and gyroscope. The biosignals were preprocessed by applying different preprocessing techniques like data filtration, data normalization, etc. The activities of the dog were predicted using CNN based algorithm. The procedure for the pet activity recognition algorithm is as follows:

A. SIGNAL DATA ACQUISITION
In order to collect the experimental data from the dogs, wearable devices were used which had accelerometer and gyroscope. These wearable devices were mounted on the neck and the tail of the 10 different dogs of the breeds, ages, and genders. 10 different activities have been examined and recorded using these sensor devices.

B. DATA PREPROCESSING AND NORMALIZATION
The raw data extracted from the sensor devices were preprocessed and filtered by applying a butter low pass filter. The filters removed the noise and unwanted signals from the data and as a result, we got refined data. Data normalization was applied to the dataset to normalize the range of all the data and bring them to the same scale.

C. DATA FORMAT AND MEASUREMENT
The wearable sensor device has a sampling rate of 33.3 Hz. Therefore, to store the data, we have a matrix with a size of 99. Every matrix is formatted in 124 x 99 =12,276 feature vectors; here 124 represents the 124 dimensions and 99 is the number of rows in each window or the number of data sizes. The first row of the vector for each element was then marked with the label for its type. The phenomenon is represented in Figure. 2 and is used for the training and testing data so that it can be used as input for the prediction model.

D. THE NETWORK ARCHITECTURE OF 1D CNN
Deep learning has been used in this study. Convolutional neural networks are the well-known deep learning approach used for different purposes like classification and detection. Deep learning architectures have been developed by researchers in different ways for different purposes. Luigi Bibbo et al. designed neural networks using a virtual reality platform [53]. Unlike the traditional Artificial neural networks, deep learning is capable of both feature extraction and classification. It automatically extracts the highly relevant features without any human intervention or handcrafted method and uses those features for classification and detection purposes. Since we used sensors' data consisting of x,y, and z values, we transformed them into vector magnitude data and this vector was used to develop 1D CNN for the classification of different activities of the dogs. 1D CNNs are among the well-known artificial neural network models used for feature extraction and classification tasks. This study investigated the activities of dogs using the 1D CNN model which comprised convolutional layers, dropout layers, flattened layers, fully connected layers, and Softmax layer.

Input layer:
The input layer of the model received six-axis data from the accelerometer and gyroscope i.e. three-axis from each sensor, in the form of vector magnitude.
Convolutional layer: The convolutional operations were used with a stride size of 1. The kernels used in the convolutional layers were 128, 128, 128, 256, and 256 while the strides were kept at 1 in each layer.
Dropout: In order to avoid overfitting and to reduce the complexity of the model, dropout layers were used while setting the dropout value to 0.5.
Output: In deep learning, activation functions play an important role in the prediction of any task. Right and wise choice of activation function results in good prediction. Rectified Linear Unit (ReLu) was used in the experiment. Since we have 10 dog activities which is a multiclass classification, therefore we used the Softmax function for the classification of all the 10 activities. Stochastic gradient descent (SGD) optimizer was applied, and the learning rate was set to 0.0001. Categorical_crossentropy was used as a loss function which calculates the loss between the actual and predicted values. The smaller the difference between the values, the higher the performance of the model. Figure 3 illustrates the architecture of the proposed model. Figure 4 illustrates the basic architecture of this study.

V. COMPLETE WORKFLOW
The overall workflow of this study is shown in figure 5. First, data related to all 10 activities were extracted from the wearable sensor devices, and at the same time videos 53158 VOLUME 10, 2022 of the respective activities were recorded and synchronized at a specific frame per second for each activity. Second, data preprocessing was conducted and noise and unwanted biosignals were removed from the dataset. A butter low pass filter was used to remove the noise. Feature engineering was performed to obtain useful information from the data while discarding unnecessary data which helped to construct an efficient algorithm. Data normalization was performed on the dataset to obtain data within the same range of values. Third, the data was split into 70 percent for training and 30 percent for testing. As the data was imbalanced, we applied data oversampling to the training dataset. The class weight technique was also applied to the training dataset. Fourth, a 1D CNN was developed and trained with the training dataset using the class-weight method. The experimental results showed that the model performed well using the class weight technique.

VI. EXPERIMENTAL RESULTS AND DISCUSSION
The experimental results are discussed in detail in this section. We conducted experiments using class weights for our class labels to balance the activities of the dog.

A. EVALUATION METHODS
The model was evaluated using different performance metrics such as accuracy, precision, recall, and ROC.
where TP represents true positive, TN is a true negative, FN is false-negative, and FP is a false positive. Precision indicates the degree of accuracy of the model in predicting the correct classification of activities. For instance, jumping was positive and all other activities of the dogs were negative. In this scenario, the correct classification of jumping divided by the sum of the correct classification and the incorrect classification of jumping gives the precision value. VOLUME 10, 2022 FIGURE 6. Confusion matrix without normalization using the test dataset.   and with normalization of all the ten activities of dogs examined in our research using class weight, respectively.

C. ACCURACY AND LOSS
The accuracy and loss of the model have been shown in figure 8 and figure 9 respectively. Figure 8 shows that when the epoch reached 700, the training accuracy increased to 99.70%. and the validation accuracy was 96.85% Likewise, the loss decreased significantly to a minimum value during the training and validation of the model for 700 epochs:

D. AUC-ROC CURVE
The AUC-ROC curve helps us to visualize the performance of our proposed model. In other words, it is an evaluation metric that shows the performance of every class while plotting the  graph between true positive rate (TPR) and false-positive rate (FPR). AUC-ROC has been shown in figure 10. The closer the graph to the left corner near 1 the better the performance of the model. The graph below shows all the curves for each class are closer to 1 which means 100% performance of the model. The AUC value of our proposed model is 100% for all the 10 classes which means our model can distinguish between the positive and negative class points correctly.  Table 3 shows the comparison of different studies, and our proposed model outperformed all the previous studies with good accuracy.

VII. CONCLUSION
This research study demonstrated the activity classification of dogs using the 1D CNN algorithm. The data was gathered from different dogs using wearable sensor devices. Two kinds of sensors i.e., accelerometer and gyroscope were incorporated into the wearable device. The data was preprocessed so that it could be used for the training of the model. The class weight approach was used to balance the data. The 1D CNN model was trained using the class weight approach.
The results were compared with the previous approaches. The experimental results showed that the class weight approach achieved higher accuracy and performance. All the ten activities i.e., walking, sitting, down, staying, eating, sideway, jumping, shaking, running, and nose work, of dogs, were predicted and classified with the highest accuracy. The overall model testing accuracy was 96.85%. This research will help improve the well-being of dogs and will provide assistance to take proactive measures for dogs' overall health.

ACKNOWLEDGMENT
(Ali Hussain and Sikandar Ali are co-first authors.)