Sound Texture Feedback for a Projected Extended Hand Interface

The ExtendedHand interface projects a computer graphics (CG) hand that synchronizes with a user’s physical hand movements onto a real environment, visually extending the user’s reach. This paper focuses on enhancing the user’s tactile perception of an object through cross-modal phenomena by providing a sound texture (auditory information that matches the object) when the CG hand touches it. Here, ExtendedHand enables users to touch objects beyond their physical reach, an experience that cannot be achieved with their physical body. In such situations, the appropriateness of adjusting sound pressure based on physical laws according to distance for users is unclear. Additionally, we have empirical knowledge that the speed at which we touch objects with our hands results in different sounds. Within ExtendedHand, since the movement of the user’s physical hand is amplified and reflected in the CG hand’s movement, the physical hand’s speed does not match the CG hand’s speed. This raises the question of whether sound texture feedback should align with the visual information of the CG hand or the proprioceptive sensory information of the physical hand. In this paper, we conducted two user studies to explore appropriate sound texture feedback for the projected CG hand. The results indicate that when the CG hand touches objects at various distances, the sound pressure should follow the same sound pressure attenuation as observed in physical phenomena. Additionally, the results suggest that despite swift tracing actions with the CG hand, users perceive sounds produced at a slower pace to be more suitable.


I. INTRODUCTION
Advancement of VR (Virtual Reality) and MR (Mixed Reality) technologies has enabled humans to possess virtual bodies distinct from their innate physical form [1], [2].One notable example is ExtendedHand, which projects a computer graphics (CG) hand into real space using a video projector, serving as a substitute for the user's own hand [3], [4].We will refer to the projected CG hand as the projected extended hand.With the help of the projector, the projected extended hand can be displayed over a wide area in real The associate editor coordinating the review of this manuscript and approving it for publication was Lei Wei .
space.This allows users to perform actions such as pointing at or touching objects that are beyond their physical reach using the projected extended hand.However, users can only see the projected extended hand on the surface of an object and cannot feel its tactile sensation.Enabling users to feel the tactile sensation of objects touched by the projected extended hand could have various applications, such as allowing them to experience the sensation of touching inaccessible objects like museum exhibits.Several studies have proposed solutions that equip users with specialized haptic devices on their hands to provide tactile feedback consistent with the contact of the projected extended hand with an object [5], [6].Although these methods can deliver FIGURE 1. Concept of perceiving the touching sensation of objects through sound texture feedback.A user can operate a projected extended hand as if it were their hand with a touch panel.When the projected extended hand overlaps on an object, the system presents a sound that matches the object to the user.The user can perceive the touching sensation of the object, even though their hand is not physically in contact with it, through the visual-audio information.
realistic tactile stimulation to the users' hands, they require the users to prepare and wear specialized devices, which limits the opportunities for use.
To provide users with tactile sensations without requiring specialized devices, one approach is pseudo-haptic feedback, which allows users to perceive tactile sensations based on visual cues [7].In ExtendedHand, our previous work proposed a method in which visual effects are added to the projected extended hand when it touches an object [8].This enables users to perceive the tactile sensation of the object.Alternatively, in VR and MR research, various methods aim to offer tactile sensations to users, not solely through visual stimuli but also through auditory cues [9], [10].Auditory stimuli can be easily presented to users using common audio devices such as headphones or speakers, making them highly applicable in ExtendedHand.
In this study, our focus is on integrating auditory feedback into ExtendedHand.Specifically, when the projected extended hand touches an object, the system presents the user with a sound that matches the object, referred to as ''sound texture.''This allows the user to experience the sensation of touching the object and perceive its tactile properties, even without haptic devices.Fig. 1 provides an overview of the method.With ExtendedHand, users can reach objects that are out of their physical hand's reach by using the projected extended hand, enabling actions that are not possible with their own body.However, in such situations, it is not immediately clear how the sound texture feedback should be governed.Two research questions arise, as illustrated in Fig. 2.
Research Question 1 addresses how to set the sound pressure of the sound texture feedback based on distance (see Fig. 2(a)).In ExtendedHand scenarios, users interact with objects at various distances, ranging from those within their reach to those beyond it.According to physical laws, the sound pressure reaching the user's ears decreases as the distance from the sound source increases.Therefore, the sound texture may become nearly inaudible when the projected extended hand touches distant objects.While adhering to physical laws, it remains unclear whether users would find this level of sound pressure natural when touching objects with the projected extended hand.Additionally, there are studies suggesting that our perception of auditory stimuli is influenced by our body image [11], [12].Thus, it is unclear whether we should directly apply the physics-based attenuation of sound pressure due to distance when users perceive, through the projected augmented hand, that they are generating sound by touching an object as a substitute for their own hand.
Research Question 2 explores whether to provide the user with a sound texture that matches the physical hand or the projected extended hand (see Fig. 2(b)).In ExtendedHand, the movements of the user's hand are amplified and reflected in the movements of the projected extended hand to facilitate interactions with distant objects.As a result, the movement of the projected extended hand becomes faster than that of the user's physical hand.When we touch objects with our hands, the generated sound varies depending on how we touch them.It is unclear whether users would perceive sound textures that align with the proprioceptive information of their hand or the visual information of the projected extended hand as more appropriate.In this paper, we will refer to the sound texture generated when tracing an object with a real hand at a speed of U [mm/s] as the ''tracing speed of the sound texture is U [mm/s].'' In this paper, we investigate the two research questions that stem from the unique characteristics of ExtendedHand, which humans have not experienced before.Specifically, we conducted experiments to determine the sound pressure level and tracing speed of sound textures based on the distance to the touched object and the movement speed of the projected extended hand.
Furthermore, several studies have reported that manipulating auditory stimuli can have a significant impact on our perception of tactile sensations related to objects [21], [22], [23], [24].A well-known example of this is the parchmentskin illusion.In this phenomenon, when users rub their hands together, the illusion of feeling a dry, parchment-like texture is created by enhancing the high-frequency components of the generated sound [25], [26].Besides, Kanek et al. reported that various factors related to button click sounds, such as sound pressure and frequency, can influence a user's perception of the weight or heaviness of the button click [10].
Based on these reports, incorporating auditory feedback into ExtendedHand used in this study is a promising approach to enhance the user's projected extended hand experience.In this research, we aim to provide users with touching sensations by presenting sounds that match the objects touched by the projected extended hand.

B. AUDITORY STIMULI AND HUMAN BODY IMAGE
Several studies have reported that auditory stimuli can influence users' perception of their body image.For instance, in the Marble-Hand illusion, when a user's hand is hit gently by a small hammer, the sound of this impact is gradually replaced with that of a hammer hitting a piece of marble.As a result, users perceived their hands as heavier and harder [27].Additionally, Tajadura et al. conducted an experiment where participants tapped a surface while progressively extending their right arm sideways.In this experiment, when sounds were generated from a location twice the distance of the tap point and presented to the participants, their perception of tactile distance increased significantly [28], [29].Furthermore, it has been suggested that this illusion can also influence actions reaching for objects farther away [30].
Vice versa, it has been suggested that the perception of auditory stimuli might be influenced by human body image.For instance, when we use cues like sound pressure to predict the distance to a sound source, there is a tendency to overestimate the distance to the source within the peripersonal space (within our arm's reach, approximately 1 m), while underestimating it in more distant spaces [11].This tendency has also been observed in the context of MR [31] and VR [32].In an experiment conducted by Serino et al. [12], participants were presented with simultaneous auditory and tactile stimuli FIGURE 3. Experimental system.An extended hand is projected from two projectors mounted on the ceiling.Users can operate the projected extended hand through the touch panel below a table.When the projected extended hand touches objects, the sound textures are played through headphones, providing users with sound texture stimuli.
and were required to respond promptly only when a specific tactile stimulus was presented.The results showed that participants responded more quickly to tactile stimuli when auditory stimuli occurred within their peripersonal space than when the sound originated from a farther space.Interestingly, the results also revealed that a brief period of using a long cane enabled participants to respond quickly to tactile stimuli when sounds were produced at the tip of the cane, which is relatively farther away.
Based on these findings, since we perceive auditory stimuli through our bodies, our body image would influence auditory perception and vice versa.In the case of the ExtendedHand, it deals with a more expanded body than what previous related research has addressed.This study aims to elucidate the appropriate manner of providing sound texture feedback concerning this extended body.

III. EXPERIMENTAL SYSTEM
We developed an experimental system specifically for two user studies described in Sections IV and V. Fig. 3 illustrates the appearance of this system.Participants could manipulate a projected extended hand on a table by moving their hand on a touch panel.Additionally, when the projected extended hand traced a target object placed on the table, a sound texture matching the object was played through headphones, providing the user with sound texture stimuli.
We constructed this experimental system using Unity 2021 on a PC (CPU: Intel, Core i7-13700, RAM: 32GB, GPU: NVIDIA, GeForce RTX 4080).Two projectors (Optoma, ML1050ST+) were ceiling-mounted to project the extended hand onto the table measuring 0.7 m × 3.0 m.To reflect the hand's movement on the touch panel to the projected extended hand, we employed the ExtendedHand system proposed by Ueda et al. [4].Users sat in a chair and manipulated the projected extended hand using the touch panel (Microsoft, Surface Pro 4) placed under the table.The delay time from touch panel input to the projected extended hand movement was 150 ms.The C/D ratio (the ratio between real hand and extended hand movements) was fixed at 5.0 for consistency throughout this study.
In this experimental setup, we assumed that the positions, shapes, and types of objects were known in advance and pre-configured this information into the system.Furthermore, we prepared sound textures by recording the sounds produced when tracing objects at varying speed and forces using a finger.When the system detected the index fingertip of the projected extended hand overlapped with an object, it played the sound resulting from applying HRTF (Head-related transform function) to the corresponding sound texture for the object through headphones (Sony, WH-1000XM3), thereby presenting the sound texture to participants.We used Google's Resonance Audio Plugin1 for the application of HRTF.We configured the position of the touch points of the projected extended hand and the participants' ears to apply HRTF.During this process, we disregarded sound reflections from objects such as tables and surrounding walls, only considering the direct path from the sound source to the participants' ears.

IV. INVESTIGATION OF SOUND PRESSURE OF SOUND TEXTURE
We conducted a user study to establish guidelines for setting the sound pressure level of sound textures based on the distance between a user and a touched object when the user traces the object with a projected extended hand.Additionally, we empirically know that a generated sound varies with the speed at which objects are traced.Therefore, we included the tracing speed of the projected extended hand as an experimental condition.
The experiments presented in this section and the next section were approved by the Research Ethics Committee of Osaka University (No. R2-28), and written informed consent was obtained from each participant.

A. EXPERIMENTAL SETUP AND PROCEDURE
The experimental setup is described in Section III.In this experiment, tufted carpets (Toli, GA1043) were used as the objects touched by the projected extended hand, as shown in Fig. 4. For the sound texture when touching the carpets, we used a sound that was recorded using a microphone (AGPTEX, Z02) while tracing the carpet with a silicone finger model (FANMAKE, QT-134).The force applied to the carpet was set at 0.4 N, and the speed of tracing the carpet was 300 mm/s.We used 300 mm/s as the common intermediate speed between the two conditions of tracing speed with the projected extended hand, which were 200 mm/s and 400 mm/s.While we considered utilizing publicly available sound datasets, we opted to create our own sound data since we specifically required sound texture corresponding to a tracing speed of 300 mm/s.Fig. 5 shows the waveform and power spectrum of the recorded sound texture.As mentioned in Section III, the participants were presented with sounds produced by applying an HRTF to the recorded sound texture.In this experiment, sound pressure attenuation due   4 with a silicon finger model.The applied force on the object was 0.4 N, and the tracing speed was 300 mm/s.Actual sound data is available at https://yushisato.com/projects/soundtexture_eh/.
to distance was turned off, allowing participants to adjust the sound pressure.To reduce exposure to ambient sounds, the noise-canceling feature of the headphones was utilized.
The procedure of the experiment was as follows: Participants received an explanation of the experiment and provided their informed consent.They then practiced operating the projected extended hand and the experimental task for 5 minutes.During the experiment, participants were required to touch the touch panel with an approximate force of 0.4 N using their index finger and operate the projected extended hand.The force applied to the touch panel was measured by a scale placed below it, with verbal feedback provided by the experimenter.Additionally, participants were required to trace a length of 150 mm back and forth along the long side of the carpet at a specified speed using the projected extended hand.To assist participants in performing these operations accurately, a red point indicating the desired movement was displayed by the system.Participants used this point as a reference while operating the projected extended hand.
After the practice session, participants performed the main experimental task, which involved the following steps: Step 1: The experimenter placed the carpet object at two distances, D i and D j , from a set of six different distances (0.5, 1.0, 1.5, 2.0, 2.5, 3.0 m).Step 2: Participants touched the objects at distances D i and D j using the projected extended hand at a fixed speed V .While doing so, a sound texture was presented through headphones.Participants set the sound pressure of the sound texture for each object to make it feel most natural when tracing the object with the projected extended hand, referencing both objects.The sound pressure levels could be adjusted within a range of 24 dB(A) to 60 dB(A) based on the position of a corresponding slider bar on a response PC.The sound pressure levels were defined as the average sound pressure of the sound texture emitted from the headphones, measured by a noise meter (Thanko, RAMA11O08) placed near the headphone sound presentation unit.
Step 3: A new carpet object was placed at a distance D k where no object had been placed previously.Participants set the sound pressure for this object using the same process as in Step 2. They could touch objects for which they had already set the sound pressure and listen to the set sound pressure.Participants adjusted the sound pressure for the object at D k while referring to the objects they had set earlier.
Step 4: Participants sequentially set the sound pressure for the remaining three distance levels among the six as the experimenter placed objects.
Steps 1 to 4 constituted one block, and participants completed six blocks, three for each of the two movement speeds (200 mm/s, 400 mm/s).In other words, participants set the sound pressure 36 times.The order of movement speeds and object placements were randomized and adjusted between participants to mitigate order effects.
After completing the main task, participants were instructed to fill out a questionnaire (see Table 1) using a 7-point Likert scale.The questionnaire included questions about Ownership and Agency and their dummy (Ownership control and Agency control) to ensure that participants felt and manipulated the projected extended hand as if it were their own [33].Sound agency questions were also included to measure whether participants perceived that the sound was generated by touching the carpets with the projected extended hand.A Sound-matching question was included to assess whether the sound textures used in the experiment matched the carpets.Additionally, a Natural-touching question aimed to gather information about participants' tactile perception experiences.Participants were also asked to verbally share their policies for setting sound pressures and provide their impressions of the sound textures.
We recruited 16 participants whose dominant hand was right and whose ages ranged from 21 to 28 (13 males and three females).The average time for each participant to complete the experiment was approximately 50 minutes.

1) MAIN RESULTS
Fig. 6 presents the result of the set sound pressure levels.We performed the two-way repeated measures ANOVA with the distance and movement speed as factors.The ANOVA result showed a significant difference in the distance factor (F(5, 75) = 79.47,p < 0.01, η 2 p = 0.84).Post-hoc multiple comparisons with Bonferroni correction revealed that farther one had a lower sound pressure in all combinations of two distances (p < 0.05).On the other hand, We did not find any significant differences in the movement speed factor (F(1, 15) = 2.97, p > 0.1, η 2 p = 0.17  7 shows the results of the calculated coefficients a and b.We tested whether the value of a was equal to the value based on the physical phenomenon, which is a = −20, for each distance.The t-test did not show a significant difference at either distance (p > 0.1).
2) SCORES FOR QUESTIONNAIRE Fig. 8 shows the evaluation results in response to the questionnaires in Table 1.We conducted a t-test for Ownership and its control category, and the results revealed a significant difference between them (p < 0.05).Similarly, we performed a t-test for Agency and its control category, and the results indicated a significant difference between them (p < 0.01).These results enhanced the credibility of the participants' survey responses.All participants scored higher than four on Sound agency, indicating that they all perceived that touching the carpets with the projected extended hand caused the sound.

3) SETTING POLICY AND IMPRESSIONS
The participants' verbal feedback at the end of the experiment was as follows: Regarding the policy for setting sound pressures, all participants except one mentioned that they    1.
adjusted the sound pressure based on the distance to the carpets.Furthermore, seven participants stated that they increased the sound pressure for faster speed.Additionally, seven participants reported setting the sound pressure higher than they would expect to generate when touching the carpet with their hands.This adjustment was done to compensate for the lack of tactile stimulation from the carpet.
Regarding their impressions of sound, nine participants mentioned that sound texture feedback enhanced their ability to perceive the sensation of touching the carpets compared to not having sound texture feedback during the manipulation practice.Furthermore, 14 participants expressed a preference for hearing sound, even when touching distant objects where sound would not typically be heard.Additionally, five participants found it challenging to determine if the projected extended hand was in contact with carpets at a distance solely based on visual information.However, with sound texture feedback, they were able to easily discern whether the projected extended hand was touching the object.

C. DISCUSSION
As intended, participants perceived sound textures being generated when they touched the carpets with the projected extended hand.The results under this condition indicated that it is appropriate for the sound pressure to decrease in a way that aligns with the physical phenomenon.This suggests that the experience of the projected extended hand does not affect the sound decay over distance.However, as the participants verbally commented, it was suggested that participants could benefit in many ways from being able to hear the sound texture.Therefore, it is indicated that, while applying distance-based attenuation within close proximity, there should be a deliberate design choice to maintain a minimum sound pressure when the distance becomes too great and the sound pressure decreases excessively.
The sound pressure levels set by the participants were generally higher than the sound produced by physically touching the carpet.As a reference, when we traced the carpet using our index finger with a force of 0.5 N and a speed of 300 mm/s, the sound pressure measured at a distance of 0.5 m was 44 dB(A).However, the average sound pressure set by the participants at the same distance exceeded 50 dB(A), as shown in Fig. 6.One possible reason, as indicated by participant comments, could be an attempt to compensate for the lack of tactile stimulation from the carpet by relying more on auditory information.This tendency to enhance another sensory stimulus in the absence of a tactile stimulus was also observed in a previous study, where tactile sensations were induced through visual effects in Extended Hand [8].
Regarding movement speed, this experiment did not detect any significant differences.Some participants commented that they increased the sound pressure when the movement speed was faster.However, upon reviewing their results, it was found that four of them had not made such settings.Based on these findings, it was considered that there is little need to alter sound pressure levels based on the magnitude of movement speed.Although the carpet was used as the target object in this experiment, future research should be conducted on a variety of objects because the characteristics of sound textures vary greatly depending on the objects.

V. INVESTIGATION OF TRACING SPEED OF SOUND TEXTURE
We conducted a user experiment to investigate the appropriate tracing speed of sound textures based on the movement speed of a projected extended hand and user-object distance.This aimed to establish a guideline for setting the tracing speed of sound texture feedback in projected extended hand experience.

A. EXPERIMENTAL SETUP AND PROCEDURE
We conducted this experiment in the same environment as Section IV, as shown in Fig. 3.In this experiment, we used a wooden board with a regular bump pattern, as shown in Fig. 9.We selected this wooden board as it allowed participants to intuitively and accurately judge the differences in sound texture tracing speeds.In this experiment, we needed to prepare sound textures at various tracing speeds.To achieve this, we traced the wooden board with a silicone finger model 27678 VOLUME 12, 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.Wooden board with a regular texture pattern used in Section V experiment.The dimensions of the board were 300 mm in width, 200 mm in length, and 9 mm in thickness, with a bump depth of 3 mm.We used two identical boards in the experiment.FIGURE 10.Sound used in Section V experiment.We recorded this sound by tracing the wooden board in Fig. 9 with a silicone finger model.(a) shows the waveform generated when the finger model passed over a single bump, and (b) is its power spectrum.We synthesized the sound when tracing the uneven board with a pattern period L = 24 mm at a speed V [mm/s] by arranging the waveform of (a) for each V /L [s].(c) is the synthesized sound when V = 500 mm/s.Actual sound data is available at https://yushisato.com/projects/soundtexture_eh/.
(FANMAKE, QT-134) at different speeds and recorded the resulting sounds with a microphone (AGPTEK, Z02).The force applied to the carpet material was 0.4 N, and the tracing speeds ranged from 50 mm/s to 600 mm/s in 5 mm/s increments.Upon analyzing the recorded sounds, we found that regardless of the tracing speed, the waveform shown in Fig. 10(a) was generated when passing over a single bump with the finger model.Therefore, in this study, we created sound textures at tracing speeds of U mm/s by arranging the unit waveform as shown in Fig. 10(a) at intervals of U /L [s], where L represents the bump's period, which was 24 mm.
We used an adjustment methodology.The experimental procedure was as follows.Participants initially received an explanation of the experiment and provided their consent.Subsequently, placed the wooden board in front of the participants and asked them to freely trace the board along its long side with their hands.We recorded participants' tracings to investigate how fast they traced the board without prior knowledge.Afterward, the participants practiced operating the projected extended hand and the experimental task for 5 minutes.Similar to the experiment in Section IV, participants were required to touch the touch panel with a force of approximately 0.4 N using their index finger and operate the projected extended hand.Additionally, they were required to trace a length of 150 mm back and forth along the long side of the wooden board at a specified speed with the projected extended hand.The system displayed a red point indicating the desired movement, and participants used this point as a reference to operate the projected extended hand.Furthermore, we monitored the force with which participants touched the touch panel using a weight scale.
After the practice session, participants repeatedly performed the main task as follows: Participants touched the wooden board placed at distance D with the projected extended hand at a specified speed V .While the projected extended hand traced the wooden board, a sound texture was presented to the participants.The tracing speed U of the sound texture was determined based on the position of a slider bar displayed on a PC.Participants set the tracing speed of the sound texture by adjusting the position of the slider bar so that they felt most natural when touching the object with the projected extended hand.The tracing speed could be set within the range of 50 mm/s to 700 mm/s, with increments of 1 mm/s.
Participants performed this task for each of the six projected extended hand movement speeds (100 mm/s, 200 mm/s, 300 mm/s, 400 mm/s, 500 mm/s, 600 mm/s) and two distances (0.5 m, 2.0 m) three times each, for a total of 36 tasks.We randomized and balanced the order of conditions across participants.
After completing the main part, participants were instructed to complete the questionnaire provided in Table 1 using a 7-point Likert scale, similar to the experiment described in Section IV.Additionally, participants were asked to verbally indicate how many times they believed a sound occurred when passing through one bump.We posed this question because, in reality, a collision sound is produced when passing from a convex to a concave of the board.However, some participants might have believed that a sound also occurred when transitioning from a concave to a convex of the board, so we inquired about their perceptions.Furthermore, participants verbally reported their policies for setting tracing speeds and their impressions of the sound textures.
We recruited 16 participants whose dominant hand was right and whose ages ranged from 21 to 30 (14 males and two females).The average time for each participant to complete the experiment was approximately 60 minutes.

1) MAIN RESULTS
We present the results of the set tracing speeds in Fig 11.Although only one collision sound was produced when a physical finger traversed through one bump on the wooden board, six participants mistakenly believed that two collision sounds occurred.Thus, we adjusted their tracing speed values by halving them.We performed a two-way repeated-measures ANOVA with the projected extended hand movement speed and distance as factors.The ANOVA result showed a significant difference in the movement speed factor (F(5, 75) = 97.91,p < 0.01, η 2 p = 0.87), the distance factor (F(1, 15) = 12.49, p < 0.01, η 2 p = 0.45), and the interaction effects (F(5, 75) = 2.49, p < 0.05, η 2 p = 0.14).Posthoc analysis of the interaction effects revealed that, under movement speeds of 300 mm/s, 400 mm/s, and 500 mm/s, the farther distance resulted in significantly greater tracing speeds (p < 0.05, Bonferroni correction).Additionally, in each distance condition, for all combinations of movement speed, except for 400 mm/s and 500 mm/s, 400 mm/s and 600 mm/s, 500 mm/s and 600 mm/s (and only for a distance of 0.5 m conditions, 200 mm/s and 300 mm/s), it was observed that higher movement speed led to faster tracing speeds of the sound texture (p < 0.05, Bonferroni correction).In the posthoc analysis of the main effect of movement speed, significant differences were observed in the same combinations as in the post-hoc analysis of the interaction effect at a distance of 2.0 m (p < 0.05, Bonferroni correction).
Next, the tracing speed U [mm/s] of sound textures when tracing objects at speed V [mm/s] with the actual hand can be expressed as U (V ) = V [mm/s].Therefore, for each participant and at distance, fitted the data of the projected extended hand's movement speed V [mm/s] and set tracing speed U [mm/s] to the equation U (V ) = cV + d and calculated the values of coefficients c and d.Fig. 12 shows the results of the calculated coefficients c and d.We tested whether the value of c was equal to the value of c = 1, which is the value when traced by an actual hand.The t-test showed a significant difference in both the distances of 0.5 m and 2.0 m (p < 0.001).

2) FREELY TRACING
We analyzed the speed at which participants freely traced the wooden board with their hands at the beginning of the experiment.We calculated the speed based on the time it took to trace a distance of 200 mm at the center of the board.Fig. 13 shows the results of the speed.The mean and standard deviation of the speed were 394±110 mm/s.
3) SCORES FOR QUESTIONNAIRE Fig. 14 shows the evaluation results in response to the questionnaires in Table 1.We performed a t-test for Ownership and its control category, and the results showed a significant difference between them (p < 0.05).Similarly, we performed a t-test for Agency and its control category, and the results showed a significant difference between them (p < 0.05).These results bolstered the credibility of the participant survey responses.All participants, except for one, scored four or higher on Sound agency.This suggests that nearly all participants perceived that they were touching  the boards with the projected extended hand, which in turn generated the sound.

4) SETTING POLICY AND IMPRESSIONS
The participants' verbal feedback at the end of the experiment was as follows: Regarding the policy for setting tracing speeds, all participants primarily relied on visual cues from the projected extended hand rather than their physical hand.Three participants mentioned the challenge of recognizing the number of bumps passed during faster movement speeds of the extended hand, which led them to adjust the tracing speed intuitively.Additionally, five participants chose a slower tracing speed of the sound texture in order to have clearer recognition of each collision sound when the movement speed of the projected hand was faster.
Furthermore, participants reported that the distance to the boards influenced their perceptions.Three participants noted that the distant board appeared to have smaller bump periods, resulting in a faster tracing speed of the sound texture.In addition, five participants reported that when tracing the distant board, they felt the need to move their actual hand more significantly in order to manipulate the projected extended hand in the indicated manner.

C. DISCUSSION
The experimental results suggested that when the movement speed of the extended hand was slow, the tracing speed of 27680 VOLUME 12, 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.1.
the sound texture should match the movement speed of the extended hand.On the other hand, when the movement speed of the extended hand was fast, it was appropriate for the tracing speed of the sound texture to be slower than the extended hand's movement speed.We interpret this result as follows: According to the participants' verbal feedback, they primarily adjusted the tracing speed of the sound texture based on the visual information of the projected extended hand.When the movement speed of the extended hand was slow, participants could easily perceive how many bumps the extended hand had crossed through visual observation.Since visual information was highly reliable, participants relied solely on visual cues to set the tracing speed.As a result, there was an approximate alignment between the movement speed of the extended hand and the tracing speed of the sound texture.
In contrast, when the projected extended hand moved quickly, participants struggled to visually distinguish the number of bumps crossed by the projected extended hand.In these situations, participants likely relied on sensory cues other than vision, such as proprioceptive senses.The C/D ratio in this experiment, the C/D ratio was 5.0, meaning that when the projected extended hand moved 50 mm, the participant's hand only moved 10 mm.As a result, participants may have felt that their hand was not moving much, leading to fewer instances of bump crossing and, consequently, fewer occurrences of the collision sound.In other words, participants' proprioceptive senses supported a slower tracing speed for the sound texture.Additionally, participants' impressions of the bump board could have provided another cue.During the free tracing task, participants traced the board with their hand at speeds ranging approximately between 350 mm/s and 450 mm/s, as shown in Fig. 13.This finding suggests that participants generally do not trace objects at speeds as fast as 600 mm/s and may struggle to imagine the sound produced at such high speeds.In such cases, participants may have preferred sounds that corresponded to tracing speeds they could more easily envision.In summary, participants' reliance on proprioceptive senses and their impressions of the bump board likely influenced their preference for slower tracing speeds when visual information was less reliable, such as when the projected extended hand moved quickly.
The experimental results also indicated that as the distance increased, the tracing speed increased, particularly between 300 mm/s and 500 mm/s.This could be attributed to the fact that as the distance increased, the bumps on the surface of the wooden board appeared smaller within the participant's field of view.While there is a phenomenon known as size constancy in object perception, it is generally acknowledged that objects are not perceived as being exactly the same size [34], [35].Participants would have felt as though they were touching objects with finer periodic patterns, potentially leading to a greater number of perceived bumps.Furthermore, in this study, we maintained a constant C/D ratio regardless of the distance.As a result, the movement of the extended hand within the participant's field of view became smaller as the distance increased.Participants subjectively felt that they moved their hands to a greater extent to achieve the indicated manipulation, which may have led them to perceive an increase in the number of bumps crossed by the extended hand.
Based on these results, it is suggested that when applying sound texture feedback to ExtendedHand, it may be more appropriate to design the tracing speed of sound textures to be proportional to the logarithm of the movement speed of the extended hand.The results also indicated that the user's field of view regarding changes in object and extended hand size varies with distance, which could potentially affect tracing speed.Therefore, when designing the tracing speed, it may be necessary to take distance into consideration.

VI. CONCLUSION
In this paper, we worked on providing users with the sensation of touching objects by presenting sound textures corresponding to the objects when the projected extended hand touched them.We focused on the unique characteristics of the projected extended hand, which amplifies the user's hand movements and allows users to touch objects that would normally be out of reach.Through user studies, we investigated how sound textures' sound pressure and tracing speed should be adjusted in such interactions.
As a result, the sound pressure level of sound textures should generally follow the same sound pressure attenuation pattern as in physical phenomena.However, even as the distance increases and the sound pressure decreases, it is advantageous to maintain a sound pressure level that remains audible to the user.The results also indicated that when the projected extended hand's speed is slow, it is appropriate to match the tracing speed of the sound texture to the extended hand's speed.However, when the projected extended hand's speed is fast, it was shown to be more appropriate to make the tracing speed of the sound texture slower than the projected extended hand's speed due to a decrease in the reliability of visual information.This research has provided fundamental and valid design guidelines for sound texture feedback when utilizing extended bodies.

FIGURE 2 .
FIGURE 2. Research questions of this study: (a) When a user touches objects placed at various distances, should the sound pressure of sound textures be lower as the distance increases?(b) In ExtendedHand, where the movement of the user's hand is amplified by a factor of K in the projected extended hand, should the occurrence of collision sounds be synchronized with movement of the user's hand or the movement of the projected extended hand?.

FIGURE 4 .
FIGURE 4. Carpet object used in Section IV experiment.The dimensions of the carpet were 250 mm in width, 100 mm in length, and 6 mm in thickness.We used six identical carpet objects in the experiment.

FIGURE 5 .
FIGURE 5. Sound used in Section IV experiment.We obtained this sound data by tracing the carpet object shown in Fig.4with a silicon finger model.The applied force on the object was 0.4 N, and the tracing speed was 300 mm/s.Actual sound data is available at https://yushisato.com/projects/soundtexture_eh/.
Fig.6presents the result of the set sound pressure levels.We performed the two-way repeated measures ANOVA with the distance and movement speed as factors.The ANOVA result showed a significant difference in the distance factor (F(5, 75) = 79.47,p < 0.01, η 2 p = 0.84).Post-hoc multiple comparisons with Bonferroni correction revealed that farther one had a lower sound pressure in all combinations of two distances (p < 0.05).On the other hand, We did not find any significant differences in the movement speed factor (F(1, 15) = 2.97, p > 0.1, η 2 p = 0.17), and the interaction effects(F(5, 75) = 1.05, p > 0.1, η 2 p = 0.07).Based on the diffusion of energy from a point source, the sound pressure P(D) [dB(A)] at a point located at a distance D [m] from a point sound source can be expressed as P(D) = −20 log 10 (D/D 0 )+P 0 [dB(A)], where D 0 [m] is the reference distance and P 0 [dB(A)] is the sound pressure at distance D 0 [m] [11].Using a reference distance of D 0 = 0.5 m, we fitted the data of distance D and set sound pressure P for each participant to the equation P(D) = a log 10 (D/0.5)+ b and calculated the values of coefficients a and b.Fig.7shows the results of the calculated coefficients a and b.We tested whether the value of a was equal to the value based on the physical phenomenon, which is a = −20, for each distance.The t-test did not show a significant difference at either distance (p > 0.1).

FIGURE 6 .
FIGURE 6. Distribution of sound pressure levels set by participants.

FIGURE 7 .
FIGURE 7. Calculated values of coefficients a and b.

FIGURE 8 .
FIGURE 8. Results of the questionnaires in Table1.

FIGURE 9 .
FIGURE 9. Wooden board with a regular texture pattern used in Section V experiment.The dimensions of the board were 300 mm in width, 200 mm in length, and 9 mm in thickness, with a bump depth of 3 mm.We used two identical boards in the experiment.

FIGURE 11 .
FIGURE 11.Distribution of tracing speeds set by participants.

FIGURE 12 .
FIGURE 12. Calculated values of coefficient c and d .

FIGURE 14 .
FIGURE 14.Results of the questionnaires in Table1.

TABLE 1 .
Questionnaire, consisting of 10 statements divided into six different categories.