From Emotions to Mood Disorders: A Survey on Gait Analysis Methodology

Mood disorders affect more than 300 million people worldwide and can cause devastating consequences. Elderly people and patients with neurological conditions are particularly susceptible to depression. Gait and body movements can be affected by mood disorders, and thus they can be used as a surrogate sign, as well as an objective index for pervasive monitoring of emotion and mood disorders in daily life. Here we review evidence that demonstrates the relationship between gait, emotions and mood disorders, highlighting the potential of a multimodal approach that couples gait data with physiological signals and home-based monitoring for early detection and management of mood disorders. This could enhance self-awareness, enable the development of objective biomarkers that identify high risk subjects and promote subject-specific treatment.


I. INTRODUCTION
H UMAN gait is mediated by a complex brain network, which relays information to the muscles via the spinal cord for locomotion. Gait characteristics constitute a clinical marker of well-being and level of activity in older adults, as well as in patients with neurological or psychiatric conditions. Impaired gait is a precursor of falls, disability and dementia [1]- [3]. Quantitative gait analysis has evolved for several decades as a clinical tool for the diagnosis and monitoring of progression of neurological disorders, such as Parkinson's disease (PD), Alzheimer disease (AD), dementia as well as mood disorders, such as depression and anxiety [4].
A key reason that gait has been implicated in several neurological and psychiatric disorders is a bi-directional interaction between the brain motor system and other cortical and subcortical structures, related to emotions and higher cognitive function [5]- [7]. Quantitative gait analysis exploits this property to develop indices of disease progression in several conditions. On the other hand, research in neuroscience and related fields indicates that improvements in posture and gait may influence brain neurotransmitters that trigger positive emotions and moods [8].
Mood disorders reflect negative feelings that last and influence brain processes related to perception, cognition and motor systems [5], [9]. According to World Health Organization (WHO), depression results in adverse health outcomes that can become severely debilitating [10]. In particular, psychomotor retardation refers to a slowing-down of both the cognitive and physical movements of the subject and it is particularly prevalent among elderly subjects with depression [11].
Mood disorders such as depression, anxiety and bipolar disorders are interleaved with other conditions such as PD, dementia and AD, presenting a complex phenotype that complicates treatment and requires far more clinical attention [12], [13]. Depression in elderly and PD patients can lead to high risk of falls and even death. Furthermore, anxiety in PD patients has been shown to be related to non-motor gait impairment [13]. This implies that anxiety and fear for falls may result in a cognitive overload that limits the walking ability of patients even when the motor brain circuits are intact. Whereas emotions are elicited by an external stimulus or event and they are short in time, moods last longer and they reflect internal states, which are not necessarily triggered by an external event [14].
With demographic shift of the aging population, there is a pressing need to seamlessly monitor gait in both elderly and patients in order to detect changes associated with mood disorders. This could allow for early psychological and/or pharmacological intervention and enable alerts to care and medical personnel. In younger adults, it can identify subjects with high propensity to develop mood disorders and allow early intervention. Furthermore, it could stimulate the development of intelligent agents and human-computer interfaces that would allow effective rehabilitation strategies to be deployed.
In this survey, we review changes in gait and body movements due to emotions and mood disorders. We focus on the clinical gaps and unmet needs along with the technological means that can be used to address these problems. Firstly, we underline the brain connectivity networks that control gait and emotions and a prominent hypothesis on how these circuits interact with each other based on evidence from lesion studies and neurological diseases such as PD. Subsequently, we explore discriminative gait and body kinematics/kinetic features and their relation to emotions and mood disorders such as depression, anxiety and bipolar disorders. We emphasize that multimodal acquisitions of gait as well as physiological data are important to link persistent emotions to mood disorders. We also highlight the need to ensure the privacy of the users while regular monitoring is enabled via a patient-centric care system, ensuring the subject to be always in control of his/her conditions. Abbreviations can be found in appendix Table V.

II. BIPEDAL GAIT AND ITS RELATION TO THE EMOTIONAL BRAIN
Bipedal gait is a cyclic complex spatiotemporal process, which is unique to humans and it has been evolutionary developed along-side higher cortical structures and human brain capabilities [5]. As shown in Fig. 1(a), several cortical brain regions such as the Prefrontal Cortex (PFC), the primary motor cortex (M1), primary/secondary somatosensory cortex (S1/S2), Supplementary Motor Area (SMA) and the Cingulate Motor Area (CMA) have been implicated during a gait cycle, gait imagination and lower limb movements [15]- [17]. Along with subcortical brain regions, such as the cerebellum, basal ganglia (BG), Pontine Nuclei (PN) and thalamus form complex brain networks that regulate gait and posture [18]- [20].
These systems are also involved in motor control and learning [18]. The consensus of evidence in the neuroscience of the motor system supports that BG forms connections with the cerebellum via the thalamus and the PN, which are regulated by a reward cortical signal [19], [21]. Furthermore, the cerebellum forms a feedback loop between the cortex, whose functional role is related to the precision of movements. Therefore, gait integrity reflects the functioning of these brain networks and their interactions. Gait abnormalities, when they are not the result of orthopedic problems and spinal cord injury, echo neurological and psychiatric disorders, such as depression, 'hysteria', anxiety, restlessness and mania [5], [22]. In the elderly, they reflect the decline of higher brain structures and circuits.
It has been shown that concurrent, cognitive tasks during walking affect gait characteristics. The so-called 'dual-task' gait performance cost is more evident in elderly and people with neurological diseases [23]. This evidence shows that gait function is not automatic and there is a link between motor control of gait and cognition [24], [25] that is strongly related to mental states and emotions [1], [5], [26]. In fact, the link between emotions and gait is bi-directional. For example, changing the way a person walks and stands could improve their response to stress [27].
The formation of emotions has been investigated for several decades in neuroscience as well as psychology and psychiatry [28], [29]. Several theories have been developed that link specific emotions to brain structures as well as several brain network hypotheses [29]- [31]. Fig. 1(b) shows the brain networks involved in the emotion formation. The limbic system, which includes the frontal lobe areas, the hypothalamus, amygdala and cingulate cortex, has been strongly associated with emotions as well as behavior and motivation [28]. Animal studies have shown that electrical stimulation of the amygdala results in increased aggression whereas when it is removed the animals do not respond adequately to fearful stimulus and sexual arousal [28], [30]. Amygdala has been also related to the expression of disgust. Both Medial Cingulate Cortex (MCC) and Anterior Cingulate Cortex (ACC) have been linked to cognitive load, whereas disgust perception has been associated with MCC Orbitofrontal, Cortex (OFC) and PFC [30].
Among the regions that play key roles in both gait motor control and emotions are the PFC and the basal ganglia [1]. PFC is located anterior of the motor brain regions and is involved in executive function, motivation and attention [28]. Its function has been also associated with reward neural pathways and addiction. The basal ganglia are strongly connected to the cortical regions such as the PFC and OFC [19], [21]. Its function is interlinked to repetitive behaviors, reward experiences and attention.
A strong body of research supports that emotions are formed via large scale cortical and subcortical brain networks and they cannot be reliably isolated to specific brain structures only [28], [29]. As illustrated in Fig. 1(b), two main neural pathways have been suggested to explain rapid body responses to 'emotional' stimulus, such as sweat and changes in heart rate and skin conductance [28]. The thalamo-amygdala route, which provides a direct response to sensory input with signs of threat and on the other hand, the thalamo-cortico-amygdala pathway, which delivers a slower response based on a more complex, corticalbased analysis of the input stimulus.
In general, there are strong evidences of bi-directional interactions between brain networks that underpin gait and emotions [1], [5], [26]. These evidences come mainly from animal studies and disease models such as PD and they highlight brain connections of the amygdala to the basal ganglia as well as the amygdala and motor cortex [1], [26]. For example, patients with PD that show symptoms of Freezing of Gait (FOG), they also show increased brain connectivity between BG and the limbic system and decreased brain connectivity between BG and cortical regions [1], [12], [32]. Patients with PD with or without FOG syndrome have also shown difficulty in recognizing emotion from facial expressions [1], [32]. Furthermore, emotional visual or auditory stimuli influences their Gait Initiation (GI) characteristics.

A. From Emotions to Mood Disorders
The etiology behind mood disorders, such as depression and bipolar, is complex and in several cases unclear [33], [34]. It normally involves both genetic as well as environmental factors, such as chronic stress [33]. These factors affect neurotransmitters, the chemical substances that control neuronal signaling [33], [34]. Subtle changes could result in increased or decreased excitation and/or inhibition, which explains the brain circuit disruption observed [29]. For example, depression is often characterized by a reduction in neurotransmitters, such as serotonin, dopamine and noradrenaline [34]. It manifests as sustained sadness and melancholia that it is not associated necessarily with external events [35]. On the other hand, bipolar disorder, also known as manic-depressive disorder, is characterized by sudden shifts between depression and mania or hypomania episodes [33]- [35]. Mania and hypomania episodes are extended periods of elevated mood that they are also associated with impatience and psychomotor agitation.
Clinical evaluations of all these conditions are subjective and they are based on self-reports, thus, subtle behavioral changes may be unnoticed [36]- [39]. It is also common for subjects with mood disorders to have different behavior at home/work compared to that in the clinic, which further complicates medical assessment [36]. Motor abnormalities that reflect the energy and/or biomechanical states of these episodes are evident and they could potentially be used in clinical practice to create objective biomarkers [37]- [39]. In fact, gross locomotion activity and gait performance have been part of patients' diagnosis for several decades. Major depression results in both motor retardation and cognitive dysfunction, which are the most prominent characteristics of disability [40], [41]. Lemke et al. [38] demonstrated that psychomotor retardation was a crucial feature of major depression. Since human gait is the combination of motor activity and cognitive involvement [7], [42], gait analysis provides a suitable exploration of psychomotor retardation in depression.
Mood disorders such as depression and anxiety can also interact and affect the progression of other diseases such as PD and dementia. For example, pharmacological treatment of anxiety with dopaminergic drugs has resulted in improved gait characteristics in PD [1]. Fear of falling among the elderly and patients with PD is very common and it is associated with anxiety and cognitive overload that limits patients' attention control abilities [13]. In fact, in elderly people, depressive symptoms increase the risk of falls and loss of independence [45], [46].
Thus far, gait performance is used in standard clinical practice to objectively evaluate and predict the risk of disability, falls and dementia in elderly [43]- [45]. Although it is more common to investigate gait impairment in older ages, young adults that suffer from mood disorders also show significant differences in gait and balance compared to control subjects [37], [46]. Gait abnormalities in young adults are not confounded by age and thus, they allow better understanding of mood disorders with relation to motor control. There is a consensus that technologies that enable monitoring of locomotion and gait performance would play a significant role in identifying early behavioral changes that could elucidate the underline reasons and shape treatment more effectively [36]- [46].

III. DISCRIMINATIVE GAIT FEATURES
Gait features are typically extracted based on the gait cycle, which is defined as the time interval between two consecutive heel strikes [3], [4]. Gait cycle can be divided into the stance and swing phases as illustrated in Fig. 2. Gait analysis normally involves measuring average basic gait cycle characteristics such as walking speed, cadence, walking base width, step length and stride length. Cadence is the number of steps per unit time, whereas walking base width is the length between midpoint to midpoint of both heels. Stride refers to the distance walked during one gait cycle. A recent survey on pervasive gait analysis explores in details gait analysis features with relation to the gait cycle [4].
A growing body of literature also supports that emotions have a significant impact on the gait initiation [47]- [50]. Gait initiation is defined as the time-period between standing motionless and walking with steady speed. This phase is divided into a movement preparation and a movement execution period [51]. Furthermore, the Center of Pressure (COP), representing the point of the ground reaction force vector, has been identified as a key measure of neuromuscular control during GI [47], [49], [50].

A. Mapping Gait Characteristics to Emotions
It is common practice to model emotions as either districtstates/categorical, continuous/dimensional or componential/ appraisal-based [14], [52], [53]. These definitions come from extensive research in psychology and neuroscience and they are inherently different. The former reflects the hypothesis that each basic emotion is formed from independent brain processes, whereas the later supports an inter-connected system.
District-states models usually include six basic feelings: happiness, sadness, fear, surprise, anger and disgust [14]. According to these models, the underline feelings are hardwired and they are common across all cultures, a concept though that has been debated. Several reviews highlighted the fact that age, gender as well as culture/language shape feelings and their intensity [14], [53]- [55]. These factors are important both for designing appropriate experiments and meaningfully interpret the results across studies. Therefore, they have been extensively discussed in the literature.
Dimensional models can have two or more dimensions. Among the most popular ones are the Circumplex, and the PAD (Pleasure, Arousal, Dominance) model [54], [55]. The Circumplex model describes feeling based on two dimensions, namely valence/pleasure and arousal. It is the most widely used space model and it has been utilized in several fields such as marketing to describe customers satisfaction and education to reflect students' attention. Fig. 3(a) shows how different emotion patterns are mapped continuously to a two-dimensional Circumplex model. This representation could link more intuitively to physiological signals that relate to arousal and valence [54], [55]. Several studies of body movements with relation to emotions also highlighted that the largest variance is encoded along the arousal dimension [56]- [58].
The PAD model, as shown in Fig. 3(b), is an extension of the Circumplex model to three dimensions, namely Pleasure, Arousal and Dominance. Further dimensions can be added to describe expectation and intensity [53], [59]. Each emotion is a linear combination of the underlying dimensions. Note that dimensions are not necessarily independent. Another advantage of using dimensional models is the ability to map emotions continuously, while minimizing the overlap between different dimensions.
The Lovheim cube of emotions relates affects directly to the major neurotransmitters that regulate brain emotional networks [60], [61], which is demonstrated in Fig. 3(c). Under this model, depressive symptoms are expressed as the inability to achieve high levels in all three neurotransmitters, namely, serotonin, dopamine and noradrenaline.
Briefly, appraisal-based models evaluate continuously both the subject's internal sentiments as well as the state of the external world. Inter-subject variability can be modeled based on this approach without the need to have a fixed number of emotions/dimensions [14]. Their use in practice though is not common because of their complexity.
Gross et al. [62] qualitatively measured the shape of the torso and reported that the expanded torso was the critical feature in representing happy walking styles. In [63], Montepare et al. revealed that gait under the influence of anger was associated with heavier foot steps compared to joyful walking styles. Roether et al. [64] found that the most discriminative features of the embodiment of emotions were the changes in the step/stride lengths and gait speed. These features were important not only to the emotional gait recognition but also to the emotional intensity evaluation. Furthermore, they also indicated that the embodiment of sadness was mainly expressed by head inclination/flexion [64]. Both happiness and anger result in increased gait speed, arm swing, vertical head movement, thigh elevation angle, step length, cadence, and movement smoothness [58], [64]. Despite that anger shares with happiness many gait characteristics including fast speed walking, happiness was rarely confused with anger [58].
The measurement of kinematics and kinetics during GI is beneficial for evaluating the relationship between the motor system and emotions [48]. Under pleasant stimuli, such as joyful videos/images, increased COP movement was observed during the anticipatory postural adjustment phase of GI [47], [49], [50]. At the same time, an increase in the velocity of the first step [47], [50] and the step length is observed [49], [50]. Similar to happiness, anger resulted in increased COP movement, gait speed, and step length, and posterior-lateral displacement of COP. During walking under sad mood, the GI reaction time of participants is reduced [47] along with the gait speed and step length [50].
GI is influenced by several factors including different feelings, disease progression and age. For example, manipulating emotional states has enhanced GI performance in patients with PD [1], [32], [48]. For example, watching threatening images resulted in increased GI speed both in PD patients and controls [48]. The authors suggested that emotional images modulated GI parameters in healthy controls and PD patients at the same level. On the other hand, Avanzino et al. [32] demonstrated that unpleasant images resulted in increased reaction time in both healthy and PD subjects, whereas unpleasant images caused reduced step size only in PD. In their recent study [1], a similar methodology was adopted on the PD patients with FOG. They observed longer reaction time and shorter step length in PD patients than controls with relation to unpleasant images.
Quantitative analysis of gait and body movements with relation to emotions and moods is an emerging research field. Several studies systematically map gait characteristics to emotions based on simple models such as the circumplex [56], [58], [62]- [68]. As listed in Table I, sadness in walkers exhibits reduced gait speed, arm swing, lateral sway, vertical head movement, step length; movement smoothness, and with smallest amplitudes of pelvic rotation, hip flexion, and shoulder flexion. On the other hand, angry and happy feelings are associated with high arousal and high dominance.

B. Gait Patterns in Mood Disorders
Several studies have been conducted to analyze the discriminative gait patterns in mood disorders using movement analysis, Table II. One of the first studies, was proposed by Lemke et al. [38] to investigate the spatiotemporal gait parameters in major depression patients. They found that depressed patients displayed significantly reduced gait velocity, stride length, cycle duration, and double limb support. Moreover, a strong correlation was observed between the cadence and the gait speed in the patients with depressive symptoms. These findings have been replicated in several studies that showed that gait patterns in depression were characterized by reduced arm swing, vertical head movement, and gait speed as well as lateral body sway and slumped posture [22], [65], [70], [74]. Dual-task paradigms were also employed to investigate the influence of concurrent cognitive tasks on gait. Both gait speed [23] and variability in gait characteristics of depressed patients while performing cognitive tasks were affected more than healthy subjects [71].
Martens et al. [13] explored the discriminative gait features in PD patients with anxiety. Statistically significant reduction in gait speed and step length, as well as the increased variability of step length and step time, were found in PD patients with higher level of anxiety. Kang et al. [37] investigated the gait characteristics in various phases of bipolar disorders (hypomanic, euthymic, depressed). Statistically significant differences were found only in hypomanic participants that exhibited increased gait speed, peak power, and peak force compared to controls. Table II also highlights studies of aging population with relation to mood disorders. Lower moods in elderly people was associated with smaller push-off force, slower gait and slumped posture compared to matched-age controls [72], [73]. Pieruccini-Faria et al. [75] demonstrated that depressive symptoms were associated with increased postural sway in Mild Cognitive Impairment (MCI), which potentially raised the risk of falls. To disentangle the effect of age on gait spatiotemporal parameters in depressed elderly, Briggs et al. [76] conducted a four-years longitudinal aging study. The study confirmed that severe symptoms of depression are related to reduced gait velocity and step length compared to age-matched controls. No statistically significant differences were found for step width and double support time.
Statistically significant differences in gait patterns have been confirmed in several studies on mood disorders. Most of them have focused on depression and several have explored depression in elderly people. Disease progression and age are confounding parameters in relating emotions and/or moods to gait characteristics as it has been shown in several studies [1], [32], [48]. Nevertheless, longitudinal studies have confirmed that several differences in gait parameters remain significant even if aging is taken into consideration [76]. Furthermore, differences in response to emotional stimulus due to disease or age are very useful to observe as they reveal underlying neuronal mechanisms that decouples emotions from disease abnormalities [32].

A. Gait Capture Methodology
Several reviews have recently focused on wearable and/or vision-based systems for gait analysis [4], [54], [77]. In particular, Stephens-Fripp et al. have reviewed gait data capture in automatic emotion recognition [54], whereas Collier et al. and others highlighted the motion measurement technologies for the psychiatric care of elderly patients [36], [78]. In this section, we briefly summarize the main principles behind these systems, as shown in Fig. 4 and outline potential future directions and challenges.
Marker-based multi-camera motion capture systems are state of the art in large clinical centers and specialized laboratories [79]- [81]. The subject is required to wear multiple sensors (reflective markers) to capture the joint movements with high precision. These systems along with force-plates [82] and electromyography (EMG) for muscle activity recordings have been the major research tools in gait analysis but they are limited to lab-use.
Ground Reaction Forces (GRFs) are measured with force plates, which are sensors placed on the floor or a treadmill to evaluate the pressure when the subject walks. The forces exerted on the floor are considered in several gait analysis studies because they are related to the load exerted to the joins [83]. The pressure insoles is an alternative wearable device also used to quantify GRFs [84], [85]. Goniometers based on flex sensors have been also used to measure the angles of different joints, such as ankles, knees or hips [86].
On the other hand, markerless, vision-based systems with RGB camera [87], [88] and RGB-D cameras [89] do not require wearing specialized equipment. In fact, robust computer vision algorithms have been developed to extract key-joints in real-time and subsequently estimate gait characteristics [90]- [92]. Although, these systems can accurately be used for gait analysis, activity recognition and energy expenditure estimation, they pose risks to the privacy of the subjects, which limit their acceptance [36].
Lately, Inertial Measurement Units (IMUs) sensors have also been successfully used to study human gait [93]- [95]. These sensors can be attached to any part of the body and use a group of accelerometers, gyroscopes and magnetometers to measure the velocity, acceleration and orientation, related to gravitational forces. To ensure patient compliance and ease of use in home environments, single sensors have been designed to acquire gait data and to detect walking gait impairment [95]. For example, the ear-worn Activity Recognition (e-AR) sensor allows patients to be continuously and discretely monitor their gait and activity levels independently. It should be noted that wearable devices require subjects to cooperate and tolerate them.
In elderly people and patients with AD or dementia, it is common to forget to wear the devices and/or to be unable to use them. Furthermore, these devices may result in further agitation for subjects with mood disorders [36]. Recently, gait characteristics have been extracted from radio signals that tract subjects and provide 3D information even through walls, the so-called 'invisible devices' [96].
Existing technology in gait and posture analysis has progressed significantly and it translates well to home use for objectively tracking psychiatric symptoms effectively. However, current clinical practice of remotely monitoring symptoms in mood disorders is based mainly on mobile text and phone messages [97]. Further research and clinical trials are required to explore new non-intrusive methodologies and innovative use of current smart sensor technologies and mobile applications.

B. Emotional Gait Databases and Arousal Methods
The availability of comprehensive datasets on emotion recognition based on gait data is scarce and comes mainly from actors. The mocap dataset only includes gait from one subject that depicts 'happy', 'sad', 'confident' and 'normal' walking [98]. Keefe et al. recorded high definition whole-body videos of actors that include four emotions: anger, fear, happiness and sadness [99]. Ma et al. presented a motion-capture library based on 30 non-professional actors [100]. Emotions encode 'angry', 'happy', 'sad' and 'neutral' states.
Typically, in most studies, to arouse the target emotions, participants are asked to imagine a situation that related to a certain emotion and to portray it during walking. Compared to the nonactor participants, professional actors have more experiences in expressing different emotions [58], [69], [79], [81], however, their body movements can be based on stereotypes and they tend to exaggerate the expression of emotions.
Other studies use auxiliary stimuli to facilitate the emotional arousal of participants. Music is one of the most common stimulus that can induce happy or sad feelings during walking [65], [82], [94]. Another method for arousing participants' emotions is visual stimuli display [50], [89], [94]. Fawver et al. [50] presented different pictures (attack, sad faces, erotica, happy faces, and neutral objects) to participants for 2-4s. Subsequently, participants were asked to start walking. In [89], [94], the affect states of participants were stimulated by watching different video clips. In these scenarios, participants have to walk immediately after the display of the video to ensure that they are influenced by the underlying emotion.

C. Recognition of Emotions by Human Observers
Previous psychological studies have demonstrated that emotional states can be successfully recognized from the gait data by human observers [62]- [64], [67], [79], [81], [101], [102] as summarized in Table III. Montepare et al. [63] conducted the first individual studies to investigate the capability of human observers to recognize emotional gait. The average recognition accuracy by observers was 78.5% (sad 94%, anger 90%, happy 74%, pride 56%) [63]. To investigate the influence of the gait speed in detecting emotions from gait, Roether et al. [64] compared emotionally expressive gait with speed-matched neutral trials. They revealed that the gait speed played a critical role in discriminating different emotional gait trials [64], which also demonstrated a strong correlation with movement activation. They also found that the human observers regarded the arm movements as an important indicator for emotion expressions.
To demonstrate an example of important gait characteristics, we collected gait data of two example participants using multiple capture methodologies (multi-camera motion capture system, ear-worn IMU sensor, RGB and depth images, and EMG sensors) that convey different emotions: neutral, happy, sad, anger, and fear. The gait cycles were determined by the heel strike events. In addition, we calculated various gait features of each gait cycle. Fig. 5 shows the characteristic 3D skeleton of different emotional gait along with several discriminative gait parameters suggested in the literature. The figure has been inspired from previous studies listed in Table I. In particular, Roether et al. that asked 25 subjects of age 25-30 years old to recall a past experience that associate with happiness, sadness, anger and fear and walk for about five meters [64]. We replicated this scenario with two volunteers to schematically depict differences in posture and how these are associated with four key human pose/gait characteristics that have been consistently reported in the literature that they link with emotions, Table I 5. Demonstration of four discriminative gait parameters commonly used in emotion detection (neutral, happy, sad, anger and fear). This is a schematic representation of human pose influenced by emotions following protocol of previous study that used healthy young/adult subjects. The subjects are asked to recall an experience associated with the feeling of happy, sad, anger and fear. 20-bin histograms describe head movement angle, IMU speed and arm swing angle, whereas the amplitude of the EMG envelope has been also estimated. In particular, head movement indicates the Range of Motion (ROM) of the head inclination angle (degrees) in the gait cycle. The histogram of the ear-worn IMU reflects the speed (mm/sec) of the head in the vertical direction. The histogram of arm swing represents the distribution of angle (degrees) between lower and upper arms. Finally, the envelopes of the rectified EMG signals are given.
have been reported in happiness and anger, whereas sadness has been associated with reduced values. The EMG sensor monitors the electrical signal of gastrocnemius and the envelope of the rectified EMG signals are demonstrated for each emotion. Furthermore, we plot the data with respect to the gait speed and stride length, which among others are key features in gait analysis, Fig. 6. We observe that the emotional walking patterns were clustered in different groups.

D. Automatic Recognition of Emotional Gait
In recent decades, machine learning-based methods for recognizing emotional gait have attracted extensive attention. We have summarized the recent studies on automatic emotion recognition from human gait in Table IV. In these studies, several common emotions (neutral, happy, sad, anger, fear, etc.) were portrayed during walking. As discussed in Section IV-D, the emotional gait data were captured by different systems, such as optical motion capture system [79]- [81], [103], RGB or RGB-D cameras [87]- [89], [104], force plates [82] and wearable smart bracelet and watch [93], [94].
We summarize the pipeline of the gait analysis methods using different sensors in Fig. 7, which can be categorized into two groups: kinematics and kinetics data. Gait kinematics is the study of the position, angles, velocities, and accelerations of human body movement. Kinetics indicate the forces, power, and energy which are the cause of individual movement [105].
To build an effective classification system, different levels of features were extracted from raw gait data. For the temporal sequences recorded from the force plate [82], EMG, IMU [94], or the accelerometers [93], a common way was to extract the statistical features from the data within a sliding window in the time domain. In addition, Zhang et al. [93] calculated the Power Spectral Density (PSD) and the Fast Fourier Transform (FFT) of the acceleration meter as the features in the frequency domain. With respect to the image data, the silhouette of the human body was typically extracted using image processing techniques [87], [104]. Subsequently, the geometric moments [87] or the local patterns [104] of the silhouette were calculated as the descriptive features. In addition, For the 2D/3D key joints captured by multi-camera motion capture system or camera [88], [89], the raw trajectories [80], [89] and the kinematic features (joint velocity [89], joint angles [79], [88], and Euclidean distance between different joints [88]) derived from joint trajectories were served as features for recognition. In [88], Chiu et al. used deep learning method to extract the 2D skeleton of the target subject from the RGB video. Furthermore, the quantitative gait parameters (gait speed, step length, cadence, and percentage of double support phase) calculated from body movements also contributed as the significant motion clues in the emotional gait recognition [79]. Since raw 3D skeleton data are typically char-    7. Overview of gait analysis methods with relation to mood disorder detection. Vision-based and IMU systems are used to measure position, velocity and acceleration-based kinematic features. Subsequently, quantitative gait parameters, such as cadence, gait speed, swing time, stride length and step length are estimated. Force plates and electromyography (EMG) data record kinetic data to relate motion parameters with forces exerted with relation to the ground and to the muscles, respectively. Time-frequency transformations along with statistical feature extraction are common ways to extract features for classification. Finally, learning-based feature extraction refers to deep neural networks methods, which automatically learn and extract relevant gait characteristics. On the other hand, bio-signals measure autonomic nervous activity, such as acute stress reflected in heart rate, sweat, blood pressure, EMG and respiration. These measurements along with neuro-physiological signals such as EEG are important to link subtle changes in human posture/gait with mental health episodes.
acterized by highly dimensional temporal sequences, Khair et al. [80] proposed a Discrete Wavelet Transform (DWT) based feature extraction method to decompose the raw data into high and low-frequency components, and Li et al. [89] used the Fourier transformation to extract 42 main frequencies and phases as the gait features. Daoudi et al. [103] calculated the covariance descriptors of raw joint trajectories, which was denoted as Symmetric Positive Definite (SPD) matrix. Particularly, each SPD matrix was regarded as a point on Riemannian manifold, and the geodesic distance between two points on the Riemannian manifold was calculated.
Prior to training the model, dimensional reduction mechanisms were usually conducted for high dimensional raw data. Venture et al. [81] used a 12 Degree-of-Freedom (DoF) model of the lower torso and variation of inclination of the trunk to represent the full body movement. Chiu et al. [88] carried out the one-way analysis of variance (ANOVA) using critical fscore to lower the dimensionality of 158 meta-features to 15. Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), and Linear discriminant analysis (LDA) are well-proven learning-based dimension reduction methods. Zhang et al. utilized PCA to perform dimension reduction, and Karg et al. [79] evaluated the performance of PCA, Kernel PCA, and LDA, respectively.
In the recognition stage, Venture et al. [81] proposed a weighted similarity measure for the joint velocity of the human body representation and used the Nearest Neighbor (NN) matching method to achieve recognition. In [80], k-NN and fuzzy k-NN methods were adopted to match the levels of coefficients decomposed by DWT, and the highest recognition accuracy was 96.07% with the first level of decomposition of DWT. The Riemannian center of mass, also referred to Karcher mean, was used for predicting the label of testing samples [103]. Except for recognition using template matching mechanism, different popular learning-based classifiers were trained in other works, such as Multi-Layer Perception [82], Self-Organized Map (SOM) [82], Support Vector Machine (SVM) [79], [87]- [89], [93], [104], Naïve Bayesian [79], [88], [89], Random Forest (RF) [88], [89], [93], [94], Decision Tree (DT) [88], [93], and Logistic Regression (LR) [88], [94]. Due to the anger and happiness gait share many movement characteristics as discussed in Section III-B, the one-versus-one classifier was trained in [89], [93]. The recognition accuracies in distinguishing anger/neutral and happy/neutral were fairly high.
Since only the raw joint data was used in [89], the highest recognition accuracy was only 55% between angry and happy emotions. In [93], highest accuracy in classifying anger and happy gait was 88.5% by training the model with the acceleration data captured from 123 participants, which emphasized that the anger and happy walking patterns can be fairly recognized by learning-based methods. Nevertheless, a large inter-subject variability was observed.
An additional challenge in automating emotional gait recognition systems is that the body expressions differ between individuals in both type and intensity. To evaluate this, the recognition results under two protocols were provided in [79], [81], [82]. For the intra-subject recognition, the train and test data are generated for the same subject. On the other hand, for inter-subject protocols, the training and test data are generated across subjects, whereas training and testing data should not include the same participants. Inter-individual differences in the expressive body movement caused inferior recognition rates compared to intra-subject (person-dependent) protocol [79], [81], [82]. Especially in [79], the highest accuracy dropped from 95% (intra-subject) to 69% (inter-subject). Despite the recognition results varied a lot in different works due to the variations of data size, emotions, training protocol, and learning methods, it can be observed that the accuracy was above 78% and 64%, for intra-subject and inter-subject protocols, respectively. Overall, the recognition rates by learningbased methods were comparative or superior to those by human observers.
In practice, automatic recognition of emotional gait is based on feature extraction, often followed by dimensionality reduction and subsequently classification. The types of features depend on the modality and they can be either extracted in time or frequency domain. Recent advances in deep neural network have made it possible to extract features in a data-driven way [106]. One of the major challenges that these methods face is the intersubject variability in both the type of posture associated with an emotion as well as the intensity. Current classification systems are comparable to human observers. The fusion of multi-modal information promises improved classification accuracy across subjects.

V. OPPORTUNITIES AND CHALLENGES
Among psychiatric and mental health conditions, mood disorders, such as depression and anxiety can occur in people across all ages and they are identified as a leading cause of disability [10]. They normally emerge as severe negative feelings that persist and they can affect the quality of life and even result in suicides and death. In young adults, these conditions are undertreated and result in reduced quality of life, decreased working performance, lower energy levels and impaired social life. In elderly people and patients with chronic diseases, depression and anxiety are prevalent and affect the progression of the disease as well as the severity of the symptoms. In particular, the ability of PD, demented and elderly people to walk could be severely affected. These result in increased risk for falls and further hospitalization.
So far, diagnoses and treatment of mood disorders are based on subjective measures and self-reports. Patients do not exhibit the full range of symptoms when they are observed by clinicians. Therefore, monitoring at home is crucial to appreciate their full clinical condition. Quantitative gait and body motion analysis have recently emerged as a powerful and objective way to diagnose and monitor mood disorders. The full potential of these technologies in-home care and patients with mild symptoms have not explored. Privacy concerns, lack of clinical validation and market regulation have hindered wide adaption of these new promising methodologies [107]. Below we highlight key challenges to address:

A) Emotions to Mood Disorders -How Do They Link?
Compared to emotions, mood disorders persist throughout life and they are associated with further symptoms. For example, depression is not only associated with sadness but also with a lack of concentration, changes in sleeping and eating patterns, fatigue and irritability. Furthermore, mild behavioral impairments, such as anxiety, irritability, euphoria and depression, they all have been associated with preclinical dementia syndromes [108]. To fully understand the evolution of the conditions, how to manage them and hinder their progression to debilitating disorders, longitudinal studies should focus on multivariate objective measures based on body movement tracking as well as biosignal monitoring.
It is important to couple body kinematics/kinetics with physiological signals such as blood pressure (BP) [97], electrocardiogram (ECG) [101], electrodermal activity (EDA) [102], heart rate (HR) [103,104], EMG [98] and respiration [105]. These signals when they are not related to activity, they reflect the emotional response of the autonomous nervous system. This response is triggered unconsciously and it is an objective measure that a subject experiences intense emotions. In fact, depression and suicidal behavior have been associated with electrodermal hypoactivity [109].

B) Availability of Datasets -Study Design
Currently, there is a profound lack of databases that link gait and body motion characteristics with emotions. Furthermore, the emotional gait data should be captured not only in different categories but also in distinct intensities. The lack of data availability and a continuous modeling of emotions hinder the application of powerful artificial intelligence techniques such as deep neural networks [19]. It is challenging to design studies that elicit consistently the same emotions with the same intensity across participants. Recent advances in augmented reality could help to elicit emotions in a more direct way. For example, gait characteristics may be modulated by displaying emotional stimulus while the subjects are walking.

C) Stigma Associated With Mood Disorders
From 1.6 million young adults with serious mental problems every year only a third seek medical/psychological care [110]. This is related to the stigma associated with these disorders, the subject's perception of the condition as a chronic, controllable disease and subject's openness to recognize the problem and seek formal help [110]. Stigma has two dimensions that relate both to the overall public perception of the disorders and the discrimination associated with this and the subjects view of public stigma. In order of the emerging technologies to be adapted widely, privacy concerns should be eliminated.

D) Emotional Gait Capturing Technology -Tolerability
Tolerability of these technologies also plays an important role, since wearing sensing equipment may result in further agitation [111]. Furthermore, elderly people and patients with severe symptoms may not be able to operate the devices, forget to wear them or do not tolerate them at all.

E) Emotional Gait Capturing Methodology -Towards Multi-Modal Acquisitions
Most of the existing studies utilize kinematic-based data and gait parameters, such as joint trajectories, joint angles and accelerations. As discussed above, the emotional gait movement is produced by the combination of the motor and cognitive system. Multi-modal data acquisition of electroencephalographic (EEG) and Near Infrared Spectroscopy (NIRS) data could shed light on the brain processes that shape emotional gait. EEG [99], [100] measures brain activity directly, from the scalp of the subject. Both EEG and NIRs have used to study gait initiation and adaptation. For example, gait initiation and precision stepping result in increased brain activity of the premotor and prefrontal regions, whereas during continuous walking cortical regions are less active [25], [112].
EEG/NIRS equipment till recently was wired and its use in naturalistic scenarios was limited. Wireless EEG/NIRS devices have been developed but their sensitivity to motion artifacts is relatively high. Therefore, further work is needed to improve artifact rejection methods and improve the signal-to-noise ratio in gait scenarios. Nevertheless, multimodal data acquisitions would play a vital role in understanding how emotions shape gait and vice-versa.

F) Automatic Mood/Gait Recognition -Towards Human-Machine Interfaces for Rehabilitation
Rehabilitation based on music and body movement has demonstrated positive outcome in several conditions such as PD and patients with brain injury. Several gait adaptation studies focus on auditory rhythms, where patients try to couple heel strikes and pacing tones, improving the gait coordination. Consequently, gait adaptation based on split-zone treadmill exercises and auditory rhythm has shown to improve gait symmetry in patients with stroke, cerebral palsy and PD [27], [29] and is an effective way to adapt stride frequency and improve gait coordination in people after stroke [30].

G) Deep Brain Stimulation and Implanted Devices
Deep brain stimulation (DBS) has shown promising results in treating along with PD, depressive symptoms, which relate to hypodopaminergic states of the disease [113]. Application of DBS at the subthalamic nucleus and ventral striatum showed improvement in both gait performance and emotional states and it has been suggested as a way to treat gait problems that link with mood disorders [12]. However, the effects of DBS are heterogeneous and depend critically on the site of delivery and the dose with relation to the underline neurotransmitters [12], [114], [115]. These evidences suggest the use of adaptive DBS solutions based on implanted devices to facilitate more targeted therapy [115], [116]. Invasive approaches involve implanted sensing devices that measure subcortical local field potentials and/or neurotransmitters such as dopamine. On the other hand, kinematic, EMG and electrocorticography data have been suggested as non-invasive biomarkers that could also drive DBS [115], [117], [118].

H) Mood Disorders in Extreme Environments
Missions in space, sea, nuclear plants and armed forces require humans to perform critical tasks in environments that are both physically and psychologically 'hostile'. For example, in long-duration space exploration, astronauts are exposed to radiation, vibration, confinement, sleep disturbances, heavy workload under extreme stress, and delayed communication with the ground. NASA has indicated that mood disorders of astronauts include depression, irritability, anxiety, and frustration. These conditions are regarded as the most severe threats to the space missions, since they compromise the ability of the astronauts to perform tasks under pressure [119], [120]. Several studies have shown that negative as well as extreme positive feelings along with cognitive workload affect behavior, decision-making and performance in extreme environments [121], [122]. In addition, sleep deprivation and decreased quality of sleep are common in space missions and can result in performance degradation and both cognitive and mental problems [120]. These are reflected in reduced response time and working memory, higher error rates and reduced visual alertness. Furthermore, these symptoms can escalate to reduced energy levels, motivation and concentration.
Mood disorders may persist after missions and result in posttraumatic stress. In particular, after space mission, detrimental effects on the musculoskeletal system due to the loss of muscle mass and muscle strength may also affect mental health and recovery [123]. Partial/micro gravity exposure has immediate effects on gait characteristics that include reduction of GRFs, stance phase duration, stride frequency, stride length, and walkto-run transition speed [124]. After a long-term exploration in space, the astronauts who completed missions within two weeks still exhibited increased variability in ankle and knee joints [125]. In summary, mental health monitoring of people living under extreme environmental conditions is essential both for the success of the missions and the well-being of the participants.

I) Automatic Emotional Gait Recognition -Beyond Mental Health
Various psychological studies indicate how expressive body and gait movements are in identifying not only emotions [79], [82] but also intended actions, gender [126], [127] and identity [128]- [130]. Compared to other modalities, gait offers the advantage that subjects can be monitored from distance and discretely. One of the potential applications for automatic emotion recognition from human gait is in the surveillance system in the public area or home-based environment.
Recently, the imitation of affective gait patterns by robots has attracted attention in order to enhance intelligent human-robot interaction [131]- [134]. A common challenge is the effective mapping mechanism from human to different robot systems with distinct mechanical and control structures.

VI. CONCLUSIONS
Mood disorders such as depression and anxiety are a leading cause of disability but often remain untreated. In young adults, mild symptoms may be undetected and ignored, partly, because of the stigma associate with these conditions. Furthermore, patients often do not demonstrate the full range of their symptoms during clinical visits.
Gait analysis and performance has been implicated in several neurological and psychiatric conditions and it is an index of well-being in both young adults and the elderly population. Recently, it has been shown that several gait and body motion parameters are related to emotions and they can be used as an objective measure to detect and evaluate the progression of mood disorders. Current technology allows for automatic recognition of emotions that can be superior to human observers. However, across subjects variability is high and it requires the development of more advanced methods based on more extensive datasets and multimodal acquisitions.
To better understand the progression of mood disorders, there is a need to link gait parameters with physiological signals such as EDA, heart rate and EEG and develop technologies that monitor gait performance/body movement at home. Furthermore, sudden changes in gait patterns may reflect mood disorders that are compound with existing conditions. Longitudinal studies that aim to understand the evolution of depression and anxiety in patients with PD and elderly will help elucidate the underlying neuronal mechanism of the pathology and allow early intervention.
Along with advances in mobile technology and intuitive apps, gait characteristics can be monitored regularly and provide an objective evaluation of mood disorders.