Finding a More Pleasant Compliance Illusion Method for a Hand-Held Device

A compliance illusion can be created if vibration bursts, which are called friction grains, are provided in response to changes in pressing force on a surface. This method has been used in many human-computer interaction applications. For instance, it can enrich virtual controls on a hand-held device with compliant feeling. One of the limitations of this method, however, is that the friction grains that it uses feel “bumpy” and “buzzing”, which may pose a hurdle to the adoption of the method by consumer products where the affective quality of tactile feedback is important. As a solution to overcome this limitation, we examined an alternative compliance illusion method, which computes the time derivative of the force applied by the user, and uses it to modulate a base vibration signal.We conducted a magnitude estimation experiment and were successful in showing that the alternative method with a sinusoidal base vibration signal can create significantly less bumpy, less buzzing, and thereby less unpleasant tactile feedback compared with the grainbased method, while achieving the same level of a compliance illusion effect as the grain-based method.


I. INTRODUCTION
The touchscreen is now the most dominant user interface for information appliances such as smartphones and tablet computers. The touchscreen affords intuitive interaction experiences to users by enabling them to manipulate interaction targets such as buttons and icons directly with fingers. On the other hand, the user experience (UX) of the touchscreen in terms of the haptic modality is relatively limited. Despite the diversity of the interaction targets on the screen, what the user feels at the fingertip is the hard glass surface, that is, the touchscreen is not compliant. Researchers have explored ideas to overcome the lack of compliance in the touchscreen interaction by exploiting a compliance illusion, for example, by providing visual, auditory, and vibrotactile compliance cues [1], [2]. Among different possibilities, the use of vibrotactile cues has the advantage of not requiring visual attention or distracting sound output. Therefore, researchers have explored methods to enhance the touchscreen interaction using vibrotactile cues, for example, by providing click feedback in response to a pressing action [3]- [8]. Although such click feedback can induce a compliance illusion of a short step motion, it cannot induce a compliance illusion for soft and elastic buttons, for which different approaches have been investigated [9]- [11].

A. THE GRAIN METHOD
A compliance illusion method that is often used in humancomputer interaction (HCI) applications, which we call the "grain method" in this study, creates a compliance illusion by providing vibrotactile bursts, which are called "friction grains" in response to the change of the force applied to a device surface by the user [9], [12], [13]. Kildal [9] reported that experiment participants were able to feel as if a hard surface that they pressed with a pen was compliant when grain feedback was provided by the pen. Kildal [13] used the same method to enable experiment participants to feel a compliance illusion when they held and squeezed a hard solid body. Since his early explorations, the grain method has been applied to many HCI applications, for example, to improve the controllability of an isometric joystick [14], to simulate a mechanical button on a touchscreen [15], to provide a buttonlike feeling on a virtual screen [16], to create an illusion of bending a controller in virtual reality (VR) [17], and to create an illusion of stepping on different ground materials in VR [18].
Despite the successful applications of the grain method, its "bumpy" and "buzzing" tactile perceptual property may pose a problem when the method is applied to consumer touchscreen devices, where the affective aspect of haptic feedback is important. The bumpy and buzzing property is inherent in the operating principle of the grain method, which is illustrated in Fig. 1. The system keeps track of the force applied by the user on an interaction surface, such as a pen tablet and a touchscreen. Whenever the force increases or decreases by a certain amount, the system generates a short burst of vibration, that is, a "friction grain". When the user continues to increase the force, he/she can feel a train of friction grains. Although the friction grains are effective for creating a compliance illusion, they create a bumpy and buzzing feeling due to their discrete temporal structure.

B. THE DF/DT METHOD
In order to find a solution to overcome the bumpy and buzzing property of the grain method, we considered an alternative approach to create a compliance illusion, which we call a modulation-based method in this study. It is not utilized in HCI applications as often as the grain mathod, but has the potential to be free from the bumpy and buzzing property because they are not based on vibrotactile grains. Visell et al. [10] and Adilkhanov et al. [11] are examples. Visell et al. [10] used an amplitude modulation of a base vibration signal in response to the change of an admittance variable, which is in a second-order dynamic relationship with an applied force. Adilkhanov et al. [11] used a frequency modulation of a base vibration signal in response to the change of an applied force to create a compliance illusion. Our modulation-based method, that we named the DF/dt method, may be regarded as a crossover of the two previous methods because it uses an amplitude modulation of a base vibration signal in response to the change of an applied force. Its operating principle is illustrated in Fig. 2. The system computes the time derivative of the force applied by the user, and uses it to modulate a base vibration signal before it is given to the user. As the vibration amplitude is proportional to the force change rate, there will be no vibration feedback when the force does not change. When the user increases or decreases the force, vibration feedback will be given to the user. The overall effect will be similar to that of the grain method, but the vibration feedback does not have a discrete temporal structure as the grain method, thereby a low possibility of a bumpy and buzzing feeling.

C. RESEARCH QUESTIONS
The design of DF/dt method was the result of many in-lab trials of different possibilities in search for an alternative compliance illusion method that feels less "bumpy" and "buzzing," whereas it is as effective as the grain method. Therefore, the two main research questions in this study were as follows.
• Will the dF/dt method be as effective as the grain method in creating a compliance illusion effect when it is applied to a hand-held device? • Will the dF/dt method create a compliance illusion with less bumpy and less buzzing feelings than the grain method when it is applied to a hand-held device? To answer these questions, we constructed a prototype system implementing the grain method and two dF/dt methods. The two dF/dt methods differ in the base signals. The first one, which we call the dF/dt-S method, uses a sinusoidal base signal, and the second one, which we call the dF/dt-W method, uses a white noise signal. We chose these two base signals as two extreme examples with regular (smooth) and random (rough) tactile properties. We expected the final tactile property of the feedback method would be influenced by that of the base signal. The details of the prototype system are presented in Section III. We then conducted a magnitude estimation experiment to compare the perceptual properties of the three methods. We describe the experiment and its results in Section IV and Section V, respectively.

II. RELATED WORK
We review different compliance illusion methods utilizing vibrotactile stimuli to position the dF/dt method that we designed for the current study. We then provide a quick review of the applications of the compliance illusion methods in order to point out the relatively low utilization of the modulation-based methods in HCI. Lastly, we quote some research efforts to change the perceptual quality of the grain method.

A. COMPLIANCE ILLUSION METHODS
Human perception of compliance is mainly through the kinesthetic and tactile channels [19]. When a fingertip is pressed on a soft surface, the finger experiences a movement, which is perceived by the kinesthetic sense. When a fingertip is pressed against a hard surface, the finger may not experience a perceivable movement but the deformation and spread of the skin, which are perceived by the cutaneous sense. Providing these direct compliance cues poses an engineering challenge, and therefore researchers have explored the possibility of using a compliance illusion.
Noting that visual, auditory, and vibrotactile stimuli accompanying a pressing action also contribute to the perception of compliance [1], researchers have explored ways to create a compliance illusion using visual, auditory, and vibrotactile feedback. For instance, Lécuyer et al. [20] showed that proper visual feedback may generate a compliance illusion. In their study, participants felt an isometric input device as a compliant one when they used it while looking at a virtual spring on the screen. Lai et al. [21] showed a possibility that sound feedback in response to a force change can create a compliance illusion on a rigid surface without using other modalities. As we mentioned earlier, there have been many studies where a short vibrotactile feedback was used to create the illusion of pressing a button [3]- [8]. In particular, recent studies have proposed vibrotactile feedback methods to create a compliance illusion for soft and elastic compliant feelings [9]- [11], which was our focus in this study.
There are two approaches to create a compliance illusion using vibrotactile stimuli. The first one is the grain method [9], [12], [13], which was already described earlier in detail. The grain method is commonly used in HCI applications as described in the next subsection. The other approach is using the modulation of a base vibration signal. Two examples using this approach are Visell et al. [10] and Adilkhanov et al. [11]. Visell et al. [10] used an amplitude modulation of a base vibration signal in response to the change of an admittance variable, which is in a second-order dynamic relationship with an applied force, to alter the perception of ground surface compliance. Adilkhanov et al. [11] used a frequency modulation of a base vibration signal in response to the change of an applied force to create a compliance illusion. As mentioned earlier, the first approach (the grain method) has the problem of "bumpy" and "buzzing" perceptual qualities. Therefore, we sought in this study an alternative to the grain method in the design space of the second approach.

B. APPLICATIONS OF COMPLIANCE ILLUSION METHODS
The grain method is more commonly adopted in HCI applications than the modulation-based method. Ahmaniemi [14] used the grain method to improve the controllability of force in the use of an isometric joystick. Kim and Lee [15] used the grain method in designing haptic feedback to simulate the feeling of a mechanical button on a touchscreen. Kim et al. [16] used a similar approach to provide a button-like feeling when a button is pushed on a virtual screen. Kim et al. [22] also used a similar approach to provide tangential and normal pseudo-force feedback on a fingertip in VR. The grain method was also used to provide a compliant response when the user applies a tangential-force operation on a touchscreen [23].
Recently, the application of the grain method is often found in VR applications. Heo et al. [17] used the same approach to create an illusion of bending a controller in VR. Strohmeier et al. [18] implemented the grain method in shoes to create an illusion of stepping on different ground materials in VR. A similar approach using grain feedback was shown to be also effective for creating a resistive illusion in a linear motion along a rail [24] or in a free motion in the air [25]. An application of the modulation-based method is presented in [11], where it was used to create the feeling of manipulating rigid or soft objects in VR.
Interestingly, the modulation-based method has been seldom used in HCI applications. Some studies focused on HCI applications [17], [26] cite Visell et al. [10], but the compliance illusion method that they actually use is the grain method. A possible reason for the choice may be that the grain method is easy to implement, for example, using an Arduino board. In the case of Visell et al. [10], it is required to simulate a second-order dynamic system, which may not be as easy as implementing the grain method. Another reason may be that the grain method is more effective in creating a compliance illusion. This is a speculation because there has been no comparison between the grain method and the modulation-based method in the literature. We are presenting the first experiment result comparing the grain method and the modulation-based method in this paper.

C. PERCEPTUAL QUALITY OF COMPLIANCE ILLUSION METHODS
Applications utilizing the grain method often report user feedback about the perceptual quality of the grain feedback. Kildal [9] shows many expressions about the feeling of friction grains provided by their experiment participants, and some of them seem to describe the nature of the bumpy and buzzing property vividly: "discrete points of vibration", VOLUME 4, 2016 "with steps," "coarse," "grainy," "distinct peaks". The tactile property of the grain train may be described as "bumpy" when the user changes the force slowly. On the other hand, the same grain train may feel "buzzing" when the user changes the force fast.
In some of the grain method applications [18], [23], [24], researchers explored ways to modify the perceptual property of the grain feedback by introducing variations and randomness to the intervals, magnitudes, and other attributes of the friction grains. Heo and Lee [23] conducted an experiment to measure the perceptual quality of the grain method in terms of the five adjectives, rough, hard, stiff, unnatural, and unpleasant, as the amplitude and frequency of the grain stimuli are varied. The vibration frequency had an effect on softness, and the amplitude variance had an effect on smoothness, naturalness, and pleasantness. No parameters had an effect on elastic feeling. Strohmeier et al. [18] created a design tool enabling rapid design of virtual materials in the space of grain parameters. They explored how the grain parameters can be optimized for generating compliance, and the effect of dynamic parameters on material experiences. However, the bumpy and buzzing property of the grain feedback is difficult to overcome because it stems from the operating principle of the grain method.

III. APPARATUS
We implemented an experimental system to be used in a user study to compare the perceptual properties of the grain method and the dF/dt method. The system consists of a handheld mock-up and a controller board. The system works in two modes, the grain mode and the dF/dt mode, which implement the two methods, the grain method and the dF/dt method, respectively. We decided to implement the two methods in the same system in order to minimize confounding effects that may be introduced by fabrication inconsistencies.
The hand-held mock-up is in the form of a small mobile device whose dimensions were 50 (W) × 80 (H) × 12 (T) in millimeters. As shown in Fig. 3, it has a force sensor (FSS1500NGT, max force = 15 N) under the circular area and two vibrotactile actuators (Taptic engine for iPhone 6s) inside on the back panel. The signal from the force sensor is amplified using an instrumental amplifier (AD8223) and is sent to the controller board.
The whole system including the hand-held mock-up and the controller is shown in Fig. 4. The Arduino board (Arduino UNO) operates differently in the two modes. In the grain mode, it implements the basic friction grain model presented in Kildal [9]. It uses a constant grain interval and a constant amplitude, A. Each grain comprises a single cycle of a 250 Hz square wave. We determined to use a square wave instead of a sinusoidal wave because it is easier to produce and may not be distinguished from a sinusoidal wave by the tactile sense [27]. In order to confirm this, we measured the actual vibration waveform of the device when actuated with a sinusoidal pulse or a square wave pulse. We were able to confirm that there was no significant difference between the  two cases as shown in Fig. 5. The grain interval in the grain method was set to 0.0625 N so that there are 80 grains in the range of 5 N, which is the same interval used by Kim and Lee [15]. The grain output from the Arduino board is amplified using a power amplifier (TDA2030A) and is sent to the two actuators. (In the grain mode, the dF/dt output from the Arduino is off.) In the dF/dt mode, the Arduino differentiates and scales the force input and sends the result to control the gain of a programmable gain amplifier (PGA) (AD820 & MCP41100). A base waveform from a laptop is modulated by the PGA and the output of the PGA is amplified by the power amplifier and sent to the two actuators in the hand-held mock-up. (In the dF/dt mode, the friction grain output of the Arduino is off.) Considering the amplifier saturation conditions of the two modes, we clipped the maximum dF/dt value at 15.625 N/s, which equals 0.0625 N/grain × 250 grains/s. The voltage input to the audio amplifier is given by where s(t) is the base signal from the personal computer (PC) (−1 ≤ s(t) ≤ 1), (dF/dt) n is the time derivative of the force divided by the maximum dF/dt value (−1 ≤ (dF/dt) n ≤ 1), and B is a constant, which should be chosen for each base signal. The constants, A and B, were chosen considering the power limit of the actuators and the balance between the perceived vibration intensities of the two modes. For base signals, we prepared two WAV files: one with a sinusoidal wave and the other with a low-passed (f c = 500 Hz, roll-off = 48 dB/octave) white noise signal. We conducted a preliminary experiment to decide the constants, A and B. We recruited 11 participants from the university (2 females, all right-handed, ages from 19 to 25, average age = 21.6) and each participant was paid 15,000 KRW (approx. 12.5 USD). To balance the perceived vibrotactile intensities, participants were asked to adjust the constant A for the grain mode, B S for the dF/dt mode with a sinusoidal signal, while the constant B W for the dF/dt mode with a white noise signal was fixed. In each session, participants repeated 18 adjustment trials (2 constants (A and B S ) × 3 B W values × 3 repetitions). As there were only two conditions, we varied the constant value, B W , in the experiment to prevent the participants from recalling and repeating their previous decisions. The three B W values were set at 100%, 75%, and 50% of the maximum value allowed  by the actuators (2.5 V). For consistency with the main experiment, a force-targeting task was used to compare the perceived intensities of two different feedback methods. The details of the force-targeting task are presented in Section IV. Participants repeated the session four times after conducting it one time as a practice. According to the results of the preliminary experiment shown in Fig. 7, we determined the constants as follows: A = 1.56 V, B S = 1.92 V, and B W = 2.5 V.

IV. EXPERIMENT
The goal of the experiment was to evaluate the tactile properties of the two dF/dt methods, one with a sinusoidal base signal (dF/dt-S condition) and the other with a white noise base signal (dF/dt-W condition), in comparison with that of the grain method (Grain condition). We chose four adjectives for this experiment: compliant, bumpy, buzzing, and unpleasant. The first three are from the two research questions of this study. We chose "unpleasant" additionally because we wanted to see whether there are correlations between the two adjectives, bumpy and buzzing, and the adjective, unpleasant. If there are high correlations between them, it may confirm that the bumpy and buzzing tactile property is indeed associated with a negative UX.

A. MAGNITUDE ESTIMATION METHOD
To estimate the four tactile properties of the compliance illusion methods quantitatively, we chose to use a magnitude estimation method [28]. We designed a magnitude estimation trial as follows. In a trial to estimate the "compliant" property VOLUME 4, 2016 of the dF/dt-S condition, for instance, a participant repeats a force-targeting task in the two conditions, dF/dt-S and Grain, freely and provides a score for the compliance of the dF/dt-S condition using a scale where the compliance of the Grain condition is assumed to be 100. In the force-targeting task, participants had to press the circular area in the handheld mock-up until the pressing force level reached a target range from 8 N to 10 N. A target range and the current pressing force were shown visually on the screen, as shown in Fig. 8(a) and 8(b). When the force level reached the target range, participants had to maintain the force in the target range for 500 ms to complete the task [14], [23]. We chose this task to enable participants to evaluate the compliance illusion methods in their typical application context. In addition, the task gave participants a chance to experience the vibrotactile feedback while they change the pressing force both fast (while pressing) and slowly (while staying in the target range).

B. PROCEDURE
We recruited 11 participants from the university (all males, all right-handed, ages from 18 to 29, average age = 22.7). Each participant was paid 15,000 KRW (approx. 12.5 USD).
Participants wore earplugs throughout the experiment to block the vibration sound from the hand-held mock-up. The experiment consisted of four sessions, one session for each of the four adjectives. The order of the adjectives was shuffled randomly for each participant. In each session, participants repeated 30 magnitude estimation trials (2 dF/dt conditions × 3 intensities × 5 repetitions). As there were only two conditions, we varied the intensity of the vibrotactile feedback in the trials (for both the dF/dt condition and the Grain condition) to prevent the participants from recalling and repeating their past trial decisions. The three intensity levels were 100%, 75%, and 50% of the maximum levels that were determined in the preliminary study for each of the three methods.
In a training session before the four main sessions, participants performed six magnitude estimation trials (2 dF/dt conditions × 3 intensities). The order of the conditions and the intensities in the training and main sessions were shuffled randomly. After the four main sessions, there was a qualitative feedback session, where participants tried the three conditions freely and described the perceptual properties of the three conditions. Fig. 8(c) shows the user interface for selecting a condition and performing a force-targeting task in the qualitative feedback session.

A. MAGNITUDE ESTIMATION RESULTS
Different participants exhibited varying response ranges in their responses. Averaging their responses as they were could result in imbalanced weights among participants. To equalize their contributions to the average results, we generated a "vector" for each session by subtracting the reference score of 100 from the scores in each trial. After constructing a vector, we normalized each vector by dividing each element by the square root of the sum of the squares as follows: where i is for participants, j is for the four sessions, k is for the trial repetitions (K = 30, 2 dF/dt conditions × 3 intensity levels × 5 repetitions), and x ref = 100. After this standardization, the sum of squares ofx ijk over the 30 trial repetitions becomes one. Fig. 9 summarizes the average scores of the magnitude estimation experiment. In the radar chart, the smaller value indicates the weaker tendency. All three conditions turned out to be similar in terms of compliance. In contrast, dF/dt-S turned out to be the least bumpy, least buzzing, and least unpleasant, among the three conditions. Fig. 10 shows the results of the experiment for each adjective. In the boxplot, the horizontal lines located on the box represent quartile 1 (25th percentile), median (50th percentile), and quartile 3 (75th percentile), respectively. The horizontal lines located on the vertical line represent the minimum and maximum values within the range 1.5 × IQR (interquartile range: quartile 3 -quartile 1) from the edge of the box. The outliers are represented as individual points.  Due to the violation of homogeneity of variance, Friedman test was used. There was no significant difference in compliance (chi-square=4.249, p=0.120). There were significant differences for other three adjectives, bumpy (χ 2 = 45.343, p < 0.001), buzzing (χ 2 = 27.931, p < 0.001), and unpleasant (χ 2 = 20.556, p < 0.001). For pair-wise comparisons, Wilcoxon signed-rank tests with Bonferroni correction were used. The dF/dt-S condition was less bumpy than the Grain condition (Z = −5.033, p < 0.001) and the dF/dt-W condition (Z = −5.587, p < 0.001). The dF/dt-S condition felt less buzzing than the Grain condition (Z = −4.330, p < 0.001) and the dF/dt-W condition (Z = −3.610, p < 0.001). The dF/dt-S condition was less unpleasant than the Grain condition (Z = −4.489, p < 0.001) and the dF/dt-W condition (Z = −3.522, p < 0.005).

B. QUALITATIVE FEEDBACK
Participants were asked to comment on the perceptual qualities of the three conditions. Three participants pointed out that the Grain condition felt bumpy when they were maintaining the force within the target range. "The vibration continues even when it reaches at the end of the press (P4)." One of the three participants considered it as an advantage: "It feels more natural because I can feel the feedback immediately when I make a slight change in the force (P11)." For the dF/dt-S condition, seven participants mentioned that it has the most pleasant feeling: "When I touched it, I wish I could touch it all the time. Its vibration is really smooth (P5)." However, three participants said the feedback was too smooth and it was difficult to maintain their force in the target range: "It was less unpleasant because there was little bumpy feeling, but it was relatively difficult to match the target range (P6)." For the dF/dt-W condition, six participants answered it was the most bumpy, and five participants said it was the most unpleasant: "It felt the most bumpy. I think it would be great to use it to indicate a warning because the feedback is intense, but it felt the most unpleasant to me (P8)." In addition, four participants responded that the feedback felt the most intense: "The vibration is too intense. However, it gives the most dynamic feeling among them. I felt like I was driving on a bumpy road in a game (P10)."

VI. DISCUSSION
We started with two research questions. Our experiment results provided positive answers to both questions. As shown in Fig. 9, the experiment results confirm that the dF/dt method with a proper base signal can create a compliance illusion as effectively as the grain method. In addition, the results show that the dF/dt method with a proper base signal can produce significantly less bumpy, less buzzing, and less unpleasant tactile feedback compared with the grain method.
The contrasting difference between the dF/dt-S method and the dF/dt-W method is interesting. We initially expected that the dF/dt method would result in less bumpy and less buzzing feelings whatever base signal is used because the dF/dt method does not use friction grains. Unlike our expectation, the dF/dt method resulted in a bumpy and buzzing feeling when a white noise signal was used as the base signal. In this case, the bumpy and buzzing feeling seems to be from the quality of the base signal. This observation is encouraging and suggests the potential of dF/dt to provide a compliance illusion with diverse tactile qualities. The sinusoidal base signal and the white noise base signal were in fact two extremes in terms of regularity or roughness. A different choice of the base signal may lead to yet another perceptual feeling. The possible range of perceptual property space that the df/dt method can cover by varying the base signal will be an immediate future research problem.
A premise in the current study was the bumpy and buzzing feeling of the friction grain is something negative and should be overcome. To support this premise with data, we included the adjective "unpleasant" in the magnitude estimation experiment. Using all answers by the participants, we calculated correlations between "bumpy" and "unpleasant" and between VOLUME 4, 2016 "buzzing" and "unpleasant", and the results were 0.84 and 0.76, respectively. The relatively high correlations seem to support our premise and encouraged us to title this paper as finding a "more pleasant" compliance illusion method.

VII. CONCLUSION
We were able to show that the dF/dt method, which is a modulation-based method, is superior to the grain method in terms of perceptual qualities, whereas it is as effective as the grain method in creating a compliance illusion with a proper choice of the base signal. The contribution of this work is two-fold. One is the design of the dF/dt method, which can be easily adopted in HCI applications as an alternative to the grain method. The other contribution is the first experimental comparison of the grain method and the modulation-based method in terms of effectiveness and perceptual qualities. We hope the dF/dt method will be utilized widely in HCI applications.