What Kinds of Robot's Touch Will Match Expressed Emotions?

This study investigated the effects of touch characteristics that change the strength and the naturalness of the emotions perceived by people in human-robot touch interaction with an android robot that has a feminine, human-like appearance. Past studies on human-robot touch interaction focused on understanding what kinds of human touches conveyed emotion to robots, i.e., the robot's touch characteristics that can affect people's perceived emotions received less focus. In this study, we concentrated on three touch characteristics (length, type, and part) based on arousal/valence perspectives, and their effects on the perceived strength/naturalness of a commonly used emotion in human-robot interaction, i.e., happiness, and its counterpart emotion, (i.e., sadness), borrowing Ekman's definitions. Our results showed that the touch length and its type are useful to change the perceived strengths and the naturalness of the expressed emotions based on the arousal/valence perspective, although the touch part did not fit such perspective assumptions. Finally, our results suggest that a brief pat and a longer contact by the fingers are better combinations to express happy and sad emotions with our robot. Since we only used a female android, we discussed future works with a male humanoid robot and/or a robot whose appearance is less humanoid.

What Kinds of Robot's Touch Will Match Expressed Emotions?
Xiqian Zheng , Masahiro Shiomi , Takashi Minato, and Hiroshi Ishiguro Abstract-This study investigated the effects of touch characteristics that change the strength and the naturalness of the emotions perceived by people in human-robot touch interaction with an android robot that has a feminine, human-like appearance.Past studies on human-robot touch interaction focused on understanding what kinds of human touches conveyed emotion to robots, i.e., the robot's touch characteristics that can affect people's perceived emotions received less focus.In this study, we concentrated on three touch characteristics (length, type, and part) based on arousal/valence perspectives, and their effects on the perceived strength/naturalness of a commonly used emotion in human-robot interaction, i.e., happiness, and its counterpart emotion, (i.e., sadness), borrowing Ekman's definitions.Our results showed that the touch length and its type are useful to change the perceived strengths and the naturalness of the expressed emotions based on the arousal/valence perspective, although the touch part did not fit such perspective assumptions.Finally, our results suggest that a brief pat and a longer contact by the fingers are better combinations to express happy and sad emotions with our robot.Since we only used a female android, we discussed future works with a male humanoid robot and/or a robot whose appearance is less humanoid.
Index Terms-Human-robot interaction, touch interaction.

I. INTRODUCTION
T OUCH is an essential factor in emotional communication for human beings in conjunction with facial expressions and language.Various human science literatures have revealed how touch interactions are used between people [1]- [3] and how touch shapes emotions [4]- [6].The positive and negative effects of touch interactions have also been broadly investigated, for instance, its importance for human well-being [7]- [9].To understand the relationship between touch and conveyed emotions, Hertenstein et al. conducted a data collection with touch-type and touched-part and different emotions (Ekman's emotions [10], [11] and prosocial emotions [12]) and proposed a touchemotional map [13].CT afferent research perspectives focused on mechanisms for processing affective touch and effects of them, e.g., researchers investigated the brain regions involved in the perceptions of CT-supported affective touch [14], [15].
Due to the advance in daily environments of socially intelligent robots that perform such services as physical/mental health support [16]- [18], education [19]- [21], and companionship [22]- [24], representing the emotions of robots is critical for achieving more natural and acceptable interactions with ordinary people.Based on this context and following the human literature related to touch in emotional communication, robotics researchers have recently started to focus on human-robot touch interaction [25]- [29].For instance, several studies scrutinized people's touches and the conveyed emotions/impressions [30]- [32].Others focused on such positive aspects of human-robot touch interaction as mental therapy or stress-buffering effects [16], [33], [34].Similar to human-human touch interaction, several studies investigated the effects of robot-initiated touches [35], [36]- [38].
However, even if these studies identified the positive effects of a robot's touch, appropriate touch design remains unknown for expressing a robot's emotions toward people.In other words, past studies generally focused on human-robot touch interaction from people to robots and overlooked touches from robots to people in emotional interaction contexts [30]- [32].Other work described a tactile-emotional map in human-human interaction [13] as well as human-robot interaction in the context of touching from people to robots [39]- [41] and reported that the touch situation contexts (e.g., emotions) complicate appropriate touch styles.Although several past studies focused on touch-speed characteristics, i.e., CT-optimal touch (around 3 ∼ 5 cm/s) [42]- [44], they did not report any detailed effects of such major touch characteristics as length and concentrated less on the design guidelines for a robot's touch interaction scheme.
One critical question remains: what kinds of touch characteristics of robots effectively match a robot's emotional expressions?When we touch another person, depending on the emotions we hope to convey, we often implicitly use different characteristics: a short/long touch, contacting/patting, and/or touching by fingers/hand.For instance, a caregiver may use touch behaviors to express emotions and/or empathy when he is interacting with seniors.If social robots are used in such situations, they must choose appropriate touch characteristics to match the emotions they are going to express.This study serves as a first step to understand the relationships between touch characteristics and emotions in human-robot touch interactions.We investigated the relationship among three kinds of touch characteristics (length, type, and part) from an arousal/valence perspective and two emotions (happiness and sadness, which is a typical emotion pair in human-robot interaction contexts to express positive and negative responses) based on the definitions of Ekman's six basic emotions [11].We experimented with an android named ERICA who has a feminine, human-like appearance (Fig. 1).Our study answers the following question: -What combinations among touch characteristics are appropriate to express happy/sad emotions?

II. TARGET EMOTIONS AND TOUCH CHARACTERISTICS
In this section, we give detailed explanations about how we designed our study, why we chose happy/sad emotions, and how we selected the three kinds of touch characteristics as well as our touch behavior implementation.

A. Robot Setup
To investigate the relationships between the expressed emotions and touch characteristics, we need a robot that can express emotion by its voice and facial expressions.It also needs enough degree of freedoms (DOFs) in its arms to touch people with different touch characteristics.Based on these requirements, we chose a robot with a human-like appearance to express emotions by facial expressions and its voice as well as its arms to touch people in various ways.
We used an android named ERICA [45], which has a feminine appearance and the capability to express human-like facial expressions.She has three DOFs in her torso and ten in each of her arms.With the two DOFs on each of her wrists and the three on her palms, she can touch people with several touch characteristics.She uses an open-loop movement control system and can update each of her actuator target positions every 50 milliseconds.Even though her silicon-based skin appears very human-like, unfortunately, her touch feels different from human skin.We put gloves on her hands to avoid mismatched impressions between her appearance and the feeling of her touch.
Fig. 1 shows ERICA's facial expressions for happy and sad emotions.We also prepared corresponding fillers and sentences to express her emotions.For instance, when she expresses a happy emotion, she laughs and says (in Japanese) "I'm really happy."Voice cues are critical for the cognitive representations of the facial expressions of emotions in adults [46].The facial expressions and voice cues are synchronized with the start timing of the touch behaviors.For the speech synthesis function, we used HOYA text-to-speech software (http://voicetext.jp/) that provides her with rich, human-like Japanese speech.

B. Touch Setting
In human-human interaction, people can touch another person to express emotions.A past study investigated the relationshipspecific maps of body regions where social touch is allowed and reported that touching the hands of another person is acceptable regardless of their relationship [47].Another study investigated the body locations touched by participants to convey different emotions and concluded that they touched the hands/forearms to express happiness and the hand/shoulders to express sadness [13].From a different perspective, another study concluded that participants mainly touched hands/forearms for both happy/sad emotions with robots [39], [40].These differences remain unclear.Since all studies suggested that touching the hand commonly expresses happy/sad emotions and that the hand is an appropriate location for social touching, we chose a participant's hand as a robot's touch target.
To guarantee that all the participants experienced the same interactions with identical touch behaviors, we positioned them next to the robot (Fig. 1), allowing them to easily observe its facial expressions as well as its touching behaviors.We placed markers on a table to indicate where the participants should put their palms and forearms.We asked them to keep their right hands and arms on those markers when ERICA did her touch behaviors.The details of the touch characteristics are explained in Section III-D.

C. Emotions: Happiness and Sadness
Emotional expression is an essential factor for social robots to build friendly relationships with interacting people.In fact, several past studies investigated the positive effects of a robot's emotional expressions, including facial expressions, body gestures, and/or speech [48]- [53].In this context, showing positive emotions (typically happiness) is one basic interaction strategy for such robots in daily environments for friendly interactions.For example, past studies designed robots that expressed happiness to build relationships with people in long-term interaction settings [54], [55].Based on these reasons, we focused on the relationships among touch characteristics with which robots more strongly and naturally express happy emotions.
We simultaneously focused on sad emotions because happy and sad emotions are considered a pair of bipolar emotions based on Russell's circumplex model of affect [56].In other words, these two emotions have opposite arouse/valence aspects (happy: high arousal and valence, sad: low arousal and valence).From the perspective of HRI design, sad emotions are typically used as negative emotions [51]- [53].In fact, expressing sadness has richer application scenarios such as showing empathy [57].We focused on just two (instead of all six) of Ekman's basic emotions for two reasons: 1) comparing all six emotions combined with all the permutations of touch characteristics would greatly complicate our analysis, and 2) happiness and sadness are the most typical emotions used in designing human-robot interactions, and the scenarios that use the remaining four are situational and less frequent.For example, anger is another candidate emotion to replace sadness based on Russell's model; however, in the context of current HRI applications, anger is probably less common than sadness.Based on these reasons, we focused on happy and sad emotion pairs in our study.

D. Touch Characteristics: Length, Type, and Part
To change the perceived strength and naturalness of the emotions expressed by a robot, we focused on touch characteristics that can show different levels of arousal/valence.For example, a touch that expresses a high arousal/valence impression is more natural for expressing a happy emotion with a stronger impression.Although many other touch characteristics resemble the reasons for selecting emotions (i.e., difficulties in comparing large numbers), we investigated three kinds of related touch characteristics: length, type, and part (Fig. 2).All three items show different arousal/valence aspects.
Length of touch: A past study reported that participants perceived a longer touch as a significantly negative valence perspective [58].On the other hand, to the best of our knowledge, no past studies in touch interaction directly investigated the effects of touch characteristics on arousal.One past study on emotional expressions for a social robot did report that a gesture's speed influences its perceived arousal, e.g., a fast behavior can express higher arousal than a slow behavior [59].Another paper [32] reported that the touch duration with high-arousal situations is relatively longer than low-arousal situations, even though the touching target is not a human being.Therefore, based on these  considerations, we prepared short and long touches that express high/low arousal/valence feelings from the length perspective.Due to a lack of published references, we conducted a small pilot study within our laboratory and heuristically decided the actual lengths for short and long touches that enable people to feel the differences between them.We used 0.5-and 2-second contact durations for the short and long touches (Fig. 3).
Type of touch: A past study reported that participants felt high arousal and valence with more pulses in the touch stimuli [58].Another work reported that a patting gesture showed more aroused emotion than contact without movement [32].We assumed that a simple contact (Fig. 3) is a single-pulse type, and a pat-like touch (Fig. 4) is a multi-pulse type.Therefore, we prepared contact and pat touches that express high/low arousal/valence feelings in the type perspective.We also heuristically determined a 50-millisecond stay time for short-pat touches and a 250-millisecond stay time for long-pat touches (Fig. 4).
Part of touch: Although people use different body parts to make contact and express meaning [27], [60], e.g., hands, elbows, upper torso, etc., they are less focused on comparing arousal/valence perspectives.However, a previous study focused on the intensity of the touch stimuli [58] and reported that a high intensity showed a greater arousal than a low intensity without any valence significance.Another past study reported that the intensity of the aroused pats is stronger than relaxed pats [32].In this study, we changed the size of area and the total touch pressure to show different touch intensities, i.e., using hand and finger (Fig. 2).

A. Hypotheses and Predictions
People can select appropriate touch characteristics that match their own emotions with which they convey their feelings more strongly and naturally and achieve smooth interaction with others.In human-robot interaction, expressing such typical emotions as happiness and sadness are also important to build smooth interactions and relationships with people.If a robot's touch behaviors match the emotions being expressed (as designed from an arousal/valence perspective), their perceived strength and naturalness (i.e., happy and sad) from people will increase.Based on these considerations, we made the following three hypotheses about the relationships between emotions and touch characteristics.
Prediction 1: When ERICA expresses a happy emotion, a short touch will be perceived as stronger and more natural than a long touch.When she expresses a sad emotion, a long touch is perceived as stronger and more natural than a short touch.
Prediction 2: When ERICA expresses a happy emotion, a pat type touch will be perceived as stronger and more natural than a contact-type touch.When she expresses a sad emotion, the contact-type touch will be perceived as stronger and more natural than the pat-type touch.
Prediction 3: When ERICA expresses a happy emotion, a finger touch will be perceived as stronger and more natural than a hand touch.When she expresses a sad emotion, a hand touch will be perceived as stronger and more natural than a finger touch.

B. Participants
Twenty-two native Japanese (11 females and 11 males whose ages ranged from 19 to 39 and averaged 29.0) participated in our experiment.They were recruited from commercially available lists to provide a wide range of backgrounds and lifestyles.None had ever interacted with an android that touched them.

C. Conditions
This study had a within-participant experiment design, i.e., each participant experienced all the combinations (16 trials) of the touch characteristics.
Emotion factor: happy and sad (Section II-C).
Length factor: short and long (Section II-D).
Type factor: contact and pat (Section II-D).
Part factor: hand and finger (Section II-D).

D. Procedure
Before the experiment, the participants were given a brief description of its purpose and procedure.This research was approved by our institution's ethics committee for studies involving human participants.Written, informed consent was obtained from all of them.First, we explained that the android expresses specific emotions with a touch, facial expressions, and speech.The participants sat next to ERICA on her left (Fig. 1).Then we calibrated the table markers to reproduce identical touch behaviors for all the participants.At the beginning of the experiment, ERICA greeted them and briefly introduced its purpose and procedure from her own perspective: thanking them for helping her collect data to improve her interpersonal-touching ability.Then she told them that she would randomly select an emotion and express it with different touch behaviors and asked the participants to compare its strength and naturalness to baseline conditions.
First, ERICA randomly selected one of the two emotions, only used facial expressions and a set of fillers and sentences to express that emotion (i.e., happiness or sadness without any touching), and identified that this was the baseline condition.
Then she randomly selected one of the eight touch behaviors and expressed an emotion by touch and the same facial expression and the set of fillers and sentences.After experiencing ERICA's emotional expression combined with a touch, the participants completed questions (Section III-E).ERICA repeated the above procedure until the participant experienced both emotions.The orders of the touch behaviors and emotions were counterbalanced.

E. Measurements
To compare and investigate the perceived emotional impressions from ERICA's emotional expressions with a touch to the baseline (i.e., without touching), we asked the participants to compare two aspects, strength ("degree of strength of the perceived emotion through the android's behaviors") and naturalness ("degree of naturalness of the touch behavior to express the emotion") on questions to the baseline condition, which we evaluated on a 1 to 7 point scale.Directly in front of the participant, we put a computer on another desk with a program that displayed a question each time ERICA finished a touch behavior.

A. Statistical Analysis about Strength Impressions
We conducted a four-factor mixed ANOVA for each scale on length, type, part, and emotion for the strength impressions.For the sphericity of the analysis, note that since the number of the levels of the repeated measures is two, sphericity has not been violated in this setting.We identified the significant main effects in the type factor (F(1,21) = 5.143, p = .034,partial η2 = .374)and in the part factor (F(1,21) = 10.337,p = .004,partial η2 = .330).We also identified the simple interaction effects between emotion and length (F(1,21) = 15,717, p = .001,partial η2 = .428)and emotion and type (F(1,21) = 22.066, p = .001,partial η2 = .512).No other simple main and interaction effects were significant (Table I).To verify our predictions, we conducted a multiple comparison of the interaction effects.

B. Statistical Analysis about Naturalness Impressions
We also conducted a four-factor mixed ANOVA for each scale on length, type, part, and emotion for the naturalness impressions.We identified the significant main effects in the part factor (F(1,21) = 49.941,p = .001,partial η2 = .704)and the simple interaction effects between emotion and length (F(1,21) = 14.384, p = .001,partial η2 = .407)and emotion and type (F(1,21) = 28.453,p = .001,partial η2 = .575).No other simple main and interaction effects were significant (Table II).To verify our predictions, we conducted a multiple comparison of the interaction effects.

C. Verification of Prediction 1: Touch Length
In our analysis of the strength and naturalness impressions, we found interaction effects between emotions and length.Therefore, we conducted multiple comparisons with the Bonferroni method of the simple main effects and identified significant differences in both the strength and naturalness impressions  (Fig. 5).For strength, we found significant differences in happy with short > long (p = .022)and sad with long > short (p = .030).For naturalness, we found significant differences in happy with short > long (p = .008)and sad with long > short (p = .006).Therefore, prediction 1 was supported; short was appropriate for happy emotions, and long was appropriate for sad emotions.

D. Verification of Prediction 2: Touch Type
In our analysis of the strength and naturalness impressions, we also found interaction effects between emotions and type.Therefore, we conducted multiple comparisons with the Bonferroni method of the simple main effects and identified significant differences in both the strength and naturalness impressions (Fig. 6).For strength, we found significant differences in happy with pat > contact (p = .012)and sad with contact > pat (p = .001).For naturalness, happy with pat > contact (p = .030),and sad with contact > pat (p = .001).Therefore, prediction 2 was supported; pat was appropriate for happy emotions, and contact was appropriate for sad emotions.

E. Verification of Prediction 3: Touch Part
In our analysis of the strength and naturalness impressions, we did not find any interaction effects between emotions and part (Fig. 7); we only found a significant difference of the effect in the part factor.Finger touches are better than hand touches for both the happy and sad emotions based on strength and naturalness perspectives (p = .004and p = .001).Prediction 3 was not supported.

F. Additional Analysis
As an additional analysis, we conducted a five-factor ANOVA by considering the gender factor.However, our results did not show any significant effects of gender.One possible inference about why there is no significant effect in the gender effects is the toucher's gender (i.e., female-appearance robot).If we conducted the same experiment with a male-type android, gender effects might be apparent due to gender combinations.
Our experiment results showed several significant effects in the touch characteristics, but not all exceeded the baseline.In other words, we did not compare the effectiveness of each touch characteristic with the baseline.We conducted a two-tailed binominal test (we classified question results into two classes: strong/natural (5 to 7) or weak/unnatural (1 to 4)) to investigate whether each touch is significantly strong or natural compared to the baseline.Our results showed similar trends to the ANOVA results.Note that detailed results about the two-tailed binominal test are described in the supplemental material.Our analysis results showed similar trends between the perceived strength and naturalness.Therefore, we investigated the correlation between strength and naturalness (happy: r = 0.69, p<.001, sad: r = 0.61, p < .001).Although these values show a positive correlation in each emotion, they are not strong (i.e., not larger than .80).

A. Design Implications
Our experimental results showed that choosing appropriate touch characteristics is helpful for designing stronger and more natural touch behaviors for robots that are expressing happy and sad emotions.At least for our robot, ERICA, which has a feminine-like appearance, short patting by fingers and longer contacting by fingers were better touch behaviors for expressing happy and sad emotions compared to contacting by hands.Even if other kinds of robots need touch interaction to show their emotions, the results suggest that touch length and types are more useful than the touch part.
Based on our data analysis, at least two touch characteristics are important: touch length and touch type.Even if hypothesis 3 is not supported, the statistical analysis showed a significant main effect in the part factor for both the strength and naturalness impressions (strength: finger > hand, p = .004,naturalness: finger > hand, p = .001).These results suggest that the touch part (i.e., finger or hand) might not be explained from the aspects of arousal/valence.But an android that is touching with her fingers might be considered more appropriate by the participants based on the results of our experiments.
In a past study that described how people conveyed their emotions to a robot by touch [39], we found several common characteristics with our study.For example, that study reported that the mean duration touch is relatively long in sad emotions.Our study also showed that a longer touch provided stronger and more natural impressions to express sad emotions.Comparing the effects of touch characteristics between touches from people and robots might be interesting to investigate how people's perceptions are different toward them.

B. Different Touch Characteristics
We focused on three kinds of touch characteristics based on arousal/valence perspectives.However, other kinds of touch characteristics should be considered.In our study, the robot's touches were relatively light for safety considerations, but applying greater strength might be one essential characteristic to express such emotions as anger.Using such other touch behaviors as gripping or stroking is another crucial factor for expressing emotions and should be implemented in future robot touch designs.
Moreover, our robot only touched the hand of the participants.Testing people's impressions when they are touched on the forearms, shoulders, or even faces would deepen our understanding of the relationships between touch characteristics and the emotions expressed by robots.Such elements as contact temperature [61] or applied pressure [62] are also critical factors that affect people's perception when they are being touched.
From another perspective, the position relationship between robots and people also have an influence.In this study, participants sat next to the robot where they could easily see its facial expressions and touch behaviors, but in real settings, people can touch others under various positional relationships.Since such relationships also limit the potential touch part and its characteristics, investigating these effects is another interesting future work.

C. Expressing Other Emotions
In this study we only focused on happy and sad emotions, because the former is often used in HRI applications [48]- [53] and the latter is an contrary emotion following Russell's definition [56].Comparing the effects of touch characteristics from arousal and valence perspectives might illuminate human-robot touch interaction designs.We did not investigate whether our knowledge is applicable for expressing the other four emotions from Ekman's basic emotions.
Social interaction scenarios for using these four emotions in current human-robot contexts are relatively less significant than the happy and sad emotion pair.However, social robots will need to express more complex feelings to interact with people in daily environments.Thus, investigating the relationships between touch characteristics and all the basic emotions is an interesting future work.

D. Gender, Appearance, and Hand Shape Effects
In this study we used an android with a feminine appearance, although human-science literature argues that perceived gender changes the touch impressions [60], [63].Since another study also reported that gender influences the perceived impressions of a robot's touch [64], we should address the effects of a robot's gender and appearance before applying our knowledge to other robots.A masculine-looking robot might increase the knowledge's usefulness.
Moreover, our android has hands that resemble human hands.Recent social robots, like Pepper, have human-like hands, but many have a more machine-like appearance and simple-shaped hands like Robovie [65].Such robots might have difficulty changing the part characteristics (i.e., hand or finger).Moreover, we put gloves on the robot's hands to avoid mismatched impressions between her appearance and the feeling of her touch, because such impressions often produce uncanny valley effects.In touch situations, researchers must consider the uncanny valley effects from movements or touch feelings as well as appearances.Even if the appearances are human-like, inappropriate movements and touch feelings easily evoke uncanny valley effects and unnatural feelings to interaction with the robot.At a minimum, touch feelings should match the assumption caused by its appearance.Based on these considerations, more knowledge about the combinations of other touch characteristics is needed for designing a robot's touch behaviors in emotional interaction.

E. Limitation and Future Work
We need to scrutinize our analysis results.Since we only used a specific android robot with a female appearance and simple contexts to express emotions from the robot to people, before generalizing our experimental results we must test different types of android robots: different appearances, including gender, age, as well as robots with more robotic appearances such as Pepper and Robovie.The knowledge from our study may only be applicable to social robots.
Moreover, how people interpret touch behaviors often reflects their social status [47], interpersonal relationships, culture backgrounds, and gender [66].In this study the robot only touched the hands of Japanese participants, i.e., the effect of other body parts and different culture backgrounds was not evaluated.Investigating these factors would be interesting to understand what touch characteristics are (not) commonly accepted in haptic communication to express emotions.Additional qualitative approaches such as interview analysis and open-question items might also support such analysis.
Another possible future work is to conduct a CT afferent study and/or using several common devices or measurements in the research field, such as EEG, NIRS, and/or fMRI, to investigate the effects of touch characteristics for useful knowledge about human-robot touch interaction.

VI. CONCLUSION
We investigated the relationship between an android's touch characteristics and its perceived strength and naturalness in the context of expressing typical emotions to an interacting person in human-robot interaction, happy and sad.Although human beings often convey emotions by touching, such knowledge is sparse for designing appropriate touch behaviors for a social robot to express emotions.Therefore, we selected three kinds of touch characteristics by considering arousal/valences and investigated their effects.
We found that touch length and touch type affected the perceived strength and the naturalness of the emotions expressed by our robot.In addition, our results suggest that a robot that touches with its fingers is deemed more appropriate for both strength and naturalness than using its hand.Based on these analyses, we concluded that a short pat and a long contact by the fingers are respectively better touch behaviors to express happy and sad emotions, at least for our robot, ERICA, who has a feminine appearance.

Fig. 5 .
Fig. 5. Average values of strength (a) and naturalness (b) of touch length and emotion factors.

Fig. 6 .
Fig. 6.Average values of strength (a) and naturalness (b) of touch type and emotion factors.

Fig. 7 .
Fig. 7. Average values of strength (a) and naturalness (b) of touch-part and emotion factors.