Vibrotactile Spatiotemporal Pattern Recognition in Two-Dimensional Space Around Hand

This study investigated vibrotactile spatiotemporal pattern recognition in the two-dimensional space around a hand. The participants placed their hands on the medium, and identified the recognized pattern presented in the medium. There were 64 rotational patterns, presented with sequential impulse vibrations. We investigated how well humans recognized the patterns presented around their hand, and identified the pattern factors (e.g., rotational direction) that affected recognition accuracy. The probability of obtaining correct answers was 48.9 %. It was observed that the start and end points, rotational direction, and the number of vibrations affected recognition accuracy. It was also found that patterns starting or ending on the ulnar side (<inline-formula><tex-math notation="LaTeX">$0^{\circ }$</tex-math></inline-formula>) of the hand were difficult to recognize, whereas those starting or ending on the distal (<inline-formula><tex-math notation="LaTeX">$90^{\circ }$</tex-math></inline-formula>) or proximal side (<inline-formula><tex-math notation="LaTeX">$270^{\circ }$</tex-math></inline-formula>) of the hand were easily recognizable. Furthermore, we found a type of the oblique effect. Patterns starting or ending in the oblique direction were more difficult to recognize than those in the cardinal direction. We also found that the clockwise rotational pattern was slightly easier to recognize than the counterclockwise rotational pattern. Finally, the underestimation of the judgment of tactile numerosity explains how the number of vibrations in the patterns affected the recognition accuracy. This result can be used as a baseline when the recognition of spatiotemporal patterns outside the body under other conditions is examined in future studies.


I. INTRODUCTION
T HERE are cases where presenting information via tactile sense is helpful. One of the assumed cases is a case where visual and audio senses are unavailable. For example, informing people in a car of the direction and distance of obstacles around the car with tactile senses would be desirable. This is because the tactile sense is often more available than the visual and auditory senses while driving [1]. This method's applicability is not limited to cars but would also be effective in cases where other audio-visual channels are overloaded and cases involving people with audio-visual disabilities. Another assumed case would be one where it is beneficial to combine tactile sense with audio or visual senses. For example, tactile senses are important to enhance realism in entertainment applications. Let us assume that an AR (augmented reality) character, visualized by stereoscopic technology, is walking around the user. By feeling the vibration while the character is walking, realism would be enhanced. In this study, we focused on vibrotactile stimulation, which can be inexpensively implemented among various types of tactile stimuli [2].
Users can recognize spatiotemporal patterns using vibrotactile stimuli. It is known that a sequence of vibrations at different locations at appropriate intervals can induce vibrotactile sensations that continuously move [3], [4]. The moving sensation using a 2D vibrotactile actuator array on the skin can present the shape of spatiotemporal patterns [5]. Such spatiotemporal patterns can provide users with symbolic information by allowing them to discriminate between patterns. In addition, spatiotemporal patterns can provide spatiotemporal information (e.g., the moving direction [6]).
To present vibrotactile spatiotemporal patterns, previous studies laid out an array of vibrotactile actuators on the body surface and caused vibrations sequentially. They have investigated how humans discriminate patterns presented at various locations in the body [5], [7], [8]. Although this approach is straightforward, some concerns exist due to the contact between the vibrotactile array and the body surface. The first concern is about the method of laying out the vibrotactile array in a limited space. If we want to present complex patterns, we need to arrange the vibrotactile array in considerably high density, but it is expensive to miniaturize and integrate the array in limited spaces. Another concern is about heat generation. The heat generated by the presentation system, including the vibrotactile array, can make it uncomfortable to use and may disturb the tactile sensation. These concerns are unavoidable when a vibrotactile array is in contact with the body.
To solve these problems, in our previous study, we investigated how well humans localized a remote vibration source in two-dimensional space via propagated vibration cues coming through a medium [9], [10]. Specifically, the participants placed their hands on a silicone rubber sheet and attempted to localize the vibration source around their hands. The localization accuracy was better with impact vibrations than sinusoidal vibrations. Although the direction of the vibration source could be considerably recognized, the distance could not be accurately recognized. We note that this method alleviates the concerns of laying out the vibrotactile actuators in a limited space and the problem of heat generation since it does not require the vibrator to contact the body.
Using the same experimental system used in our previous study [9], the present study investigated the human recognition ability towards vibrotactile spatiotemporal patterns in the two-dimensional space around the hand (see Fig. 1). Specifically, we used eight vibrotactile actuators around the hand and made participants recognize 64 patterns (see Fig. 2(a) and (b)). Because we found in a previous study that humans can recognize the direction of the remote vibration source [9], we defined the patterns as rotational ones. Such patterns can be recognized with directional recognition of the vibration source, instead of linear patterns. As a result of the experiment, we could explicitly characterize a human's ability to recognize rotational patterns in terms of various factors of patterns comprehensively (e.g., the rotational direction or starting point of patterns). The results of the present study provide basic information on the human ability to recognize spatiotemporal patterns around the hand and insights into the development of spatiotemporal pattern presentation displays.

II. RELATED WORK
As related work, we introduce previous studies that used remotely distributed vibration sources from the human body to investigate the human localization ability. Subsequently, we introduce studies on the recognition of vibrotactile spatiotemporal patterns presented on the surface of the body.

A. Vibrotactile Localization Outside Body Surface
Some studies have investigated localization ability when users are holding the edge of a one-dimensional medium (e.g., a stick), which is stimulated at a certain distance from the point it is held. Miller et al. had participants hold a one-dimensional medium (such as a stick) in their hand and actively shake it to determine whether they could estimate the point of contact when striking an object [11], [12]. Sreng et al. had participants hold one end of a stick in each hand and use several vibration patterns to estimate the location of the vibration source [13]. In contrast to these studies, which used a one-dimensional medium, we used a two-dimensional medium. This is because more patterns can be represented in a twodimensional space than in a one-dimensional space.
The study by [14] proposed a system to present users with a vibration source in a two-dimensional space, but it did not clarify how well users could localize the source because they did not conduct an experiment. In similar settings, our previous study examined the localization ability of a hand placed on a two-dimensional medium when an impact was made somewhere on the medium around the hand [9]. We found that the participants could not recognize the distance, but recognized direction to some extent. We found variability in the absolute error of direction recognition depending on the stimulus direction. We also found a directional bias, which indicated a significant difference in perceived direction and stimulus direction depending on the stimulus direction. Since recognizing direction is possible to some extent, the present  study used eight different vibrotactile stimuli directions from those used in our previous study [9] to investigate vibrotactile spatiotemporal pattern recognition.
Previous studies have reported that specific factors of rotational patterns affect the accuracy of pattern recognition. Regarding the rotational directions (clockwise (CW) or counterclockwise (CCW) directions) of stimuli, some studies reported a significant effect of the rotational directions of stimuli on accuracy, while others did not. Some also reported that the accuracy is better in the case of CW than in the case of CCW patterns [17]. In their experiment, eight out of ten participants were right-handed, and it was unclear whether the interaction effect of the dominant hand and rotational direction was significant. Conversely, other studies reported that rotational directions do not affect the accuracy of recognition [16], [23], [25], [39]. It should be noted that while there was no significant difference in accuracy depending on the stimuli's rotational direction in the experiment [25], some participants reported a difference in constructing mental models for CW and CCW stimuli. We investigated the effect of rotational direction on the accuracy of our experimental setup.
Regarding the starting point of rotational patterns, according to the results of Seo et al. [23]'s experiment, it is known that the starting point significantly affects recognition accuracy. In their experiment, the starting point on the ulnar side was significantly better than that on the distal, proximal, and radial sides. In contrast to the study by Seo et al. [23], in our previous study [9], the absolute directional error of vibrotactile localization was worse on the ulnar side and better on the distal one. The characteristics of the absolute directional error of vibrotactile localization are related to spatiotemporal pattern recognition in terms of the starting point in the present study. It was interesting to see how these contrasting results would relate to the present study's results. In addition, we assumed that the end point would affect the accuracy, similar to the starting point.
Furthermore, we hypothesized that the effect of the start and end points might be related to the "oblique effect." Although this term has been used in many different experimental settings, the basic idea is that the performance with oblique stimuli is worse than that with stimuli oriented in the cardinal directions (i.e., horizontal and vertical). For example, Appelle and Gravetter [40] confirmed the oblique effect, in which participants were less accurate when discriminating lines oriented obliquely than those oriented either vertically or horizontally. Recently, Kappers et al. [41] observed an oblique effect in vibratory direction recognition on the back. Their results showed that both the bias and variance of direction recognition were worse in the oblique direction than in the cardinal direction. It was hypothesized that the recognition of spatiotemporal patterns characterized by the cardinal direction (starting and ending in the cardinal direction) would be more accurate than those characterized by the oblique direction.
In addition, the number of vibrations in the stimuli also affects the accuracy in recognizing spatiotemporal patterns. Humans often underestimate the number of vibrations in spatiotemporal vibrational patterns [42]. If humans recognize such patterns using the numerosity cue, the recognition accuracy would be worse when the number of vibrations in the stimuli is large. Indeed, it has been reported that the number of vibrations directly affects the recognition accuracy of spatiotemporal patterns [43], although studies provided vibrotactile rotational patterns on the hand directly.
All previous studies have investigated the recognition of spatiotemporal patterns that directly present vibration on body surfaces. Yet, no study has explicitly investigated spatiotemporal pattern recognition stimulated by vibrotactile actuators outside the body, which the present study investigates. We were interested in how factors such as the starting point, end point, rotational direction, and number of vibrations would affect the recognition accuracy of spatiotemporal patterns in a two-dimensional space around the hand.

A. Participants
Fourteen participants (13 males and one female, all righthanded, with a mean age of 22.4 (SD: 1.1) years) took part in the study. The average height and width of the participants' hands were 18.14 (SD: 0.55) cm and 17.26 (SD: 1.12) cm, respectively. All participants were naive to the purpose of the study. Ethical approval for this study was obtained from the ethics committee of the University of Electro-communications (approval number: 21052).

B. Apparatus
The apparatus used in this experiment was the same as that used in our previous study [9]. Please refer that previous study for the details. Fig. 3 shows the appearance of this apparatus.
Participants were seated comfortably on a chair and instructed to close their eyes during the task. They wore noise-canceling headphones that played white noise to muffle the external sounds. The right arm was placed on the armrest and their hands were set in the designated area on a silicone rubber sheet (see the designated hand position in Supplementary Fig. 1). The size of the silicone rubber sheet was 500 mmÂ500 mm, and its thickness was 10 mm. The hardness of the silicone rubber sheet was 30 (tested using a Shore A durometer). This sheet was placed on soft urethane foam (INOAC CORP, ECZ, 25 % ILD was 80) to diminish the influence of propagating vibration propagation from the ground. The size of the soft urethane foam was 500 mmÂ500 mm, and its thickness was 100 mm. Solenoids (CBS10290100, TAKAHA KIKOU Co.) were placed inside the urethane foam. The diameter of the contactor at the top of the solenoid was 3 mm. The solenoids were actuated using a driver circuit (SB-6565-01, TAKAHA KIKOU Co.).
To characterize the vibration, we measured the displacement of the silicone rubber sheet indented by the solenoid contactor by using a laser displacement sensor (ZX2-LD50, OMRON Co.). The maximum displacement was approximately 1 mm and the duration was approximately 0.2 s at the point of contact with the solenoid contactor. See Fig. 7. in our previous paper [9] for acceleration at 0 mm and 130 mm from the vibration source to determine the damping when the vibration propagates. Additionally, see Fig. 8 in our previous paper [9] for the power spectral density of acceleration.

C. Stimuli
The key design factors for spatiotemporal patterns were the shape of the patterns and the stimulus onset asynchrony (SOA). 1) Pattern Shape: As described in the Introduction, we used rotational patterns. Though the linear patterns such as moving away from or approaching a hand were other candidate patterns, humans cannot recognize the distance accurately [9] and thus, it is difficult for participants to recognize such linear patterns. In contrast, it was expected that humans can recognize the rotational patterns with the human ability to recognize direction better. Thus, we decided to use rotational patterns in this experiment.
To clearly characterize the human recognition ability for specific spatiotemporal patterns around the hand, we wanted to increase the difficulty of this task. If the tasks were too easy, it would be difficult to observe the difference in recognition accuracy due to the stimulus factor. We assumed that the difficulty of the task depended on the number of spatiotemporal patterns presented to the subjects.
To define the appropriate set of spatiotemporal patterns, we set up a preliminary experiment with 32 spatiotemporal patterns made up of only four points (0 , 90 , 180 , and 270 ) and 11 participants (see Supplementary Fig. 2 for the patterns). In the preliminary experiment, participants were presented with patterns and they identified 32 candidates. Consequently, the recognition accuracy was found to be 84.1 % and the maximum information transfer (IT) was 4.24. Refer to Supplementary Note 1 for details on the task, and results. We found that there was a significant main effect of the starting point but not for other factors such as rotational direction or end point of patterns.
In the main experiment, we presented participants with greater number of patterns. Eight points were indented by the solenoid's contactors in eight directions (see orange points in Fig. 2(a)). The distance between the points from the center of the sheet was 130 mm. Fig. 2(b) shows the 64 spatiotemporal patterns that were used. The patterns were defined by a combination of rotational direction, starting point, and end point. In addition to these three factors, the number of vibrations also characterized the patterns. The rotational direction levels were CW and CCW. The levels of starting and end points were 0 , 45 , 90 , 135 , 180 , 225 , 270 , and 315 . The numbers of vibrations were three, five, seven, and nine.
2) SOA Optimization: The current experiment elucidates the effect of spatial factors of patterns (e.g., starting point of patterns). On the other hand, stimulus onset asynchrony (SOA), or interval time between the onsets of subsequent stimuli, might affect the recognition of patterns. To clearly focus on the spatial factors, we optimized the SOA, which is a temporal factor, prior to the main experiment. We conducted another preliminary experiment to define SOA to make the recognition of the vibrotactile patterns better. In this preliminary experiment, participants adjusted the SOA for all 64 patterns. We asked the participant to determine the SOA, where the vibrotactile pattern was clearly recognizable. The same SOA was assigned to all the intersolenoids in one pattern. In total, there were 64 trials. In one trial, the SOA was adjusted for one of the 64 patterns. The participants received the vibration on their right hand and could change the SOA value by pressing the keyboard with their left hand's finger. Once the SOA value was changed, corresponding vibrations would be presented. The change of SOA and the presentation of vibration during one trial could be repeated without limit. We hypothesized that the optimized SOA depends on the aforementioned four factors (rotational direction, starting point, end point, and number of vibrations).
To verify this, we performed a generalized linear model (GLM) analysis. The GLM was fitted to regress the optimized SOA values using four factors. Since the optimized SOAs were positive continuous values that were not expected to have a normal distribution, the GLM employed a logarithmic link function with a gamma distribution. Based on the likelihood ratio test (Type II test), a statistically significant main effect of the number of vibrations was found [df ¼ 3; x 2 ¼ 10:787, and p ¼ 0:012] (see Fig. 4. The other three factors had no statistically significant main effects (p > 0:05). For post-hoc tests of the significant main effect of the number of vibrations, we conducted multiple comparisons. The significance level was set at 0.05 with Bonferroni correction. There were statistically significant differences between patterns 3 and 9 only in terms of the number of vibrations in the patterns. Based on the results of the preliminary experiment, the optimized SOA was found to significantly depend only on the number of vibrations. Thus, we configured the SOA with regard to the number of vibrations for each spatiotemporal pattern.

D. Task
At the beginning of the main experiment, the participants were presented with written instructions that described the situation and tasks of the experiment. After reading it, the participants moved on to the experiment.
In one trial, one of the 64 patterns was presented to each participant. The participants answered questions regarding the starting point, end point, and rotational direction. Thus, they could answer one of 128 parameter combinations in total (= 8 starting points Â 8 end points Â 2 rotational directions). Because we wanted to investigate the absolute recognition accuracy of each spatiotemporal pattern, we did not make participants select the patterns among the 64 pattern candidates. If we made participants discriminate the patterns, the result of discrimination would be influenced by the set of patterns, which should be avoided.
Each block consisted of 64 trials. The order of presentation of the patterns in each block was randomly assigned. There were four blocks per participant, resulting in 256 trials conducted per participant. To reduce their workload, the blocks were assigned to two consecutive days per participant. Two blocks were assigned to the first day and the other two blocks were assigned to the second day. There was a 10-min break between blocks in a day. Fig. 5 shows the recognition accuracy for each pattern. The average accuracy was 48.9 %. In our experiment, we let the participants choose the parameters (starting points, end points, and rotational directions) as answers. The number of combined parameters was 128. Among these combinations, there was one correct answer that was the same as the pattern ID that had been presented to the participant tactilely. The probability that the correct answer will be chosen by chance is 1/128 = 0.78 %. As compared to this value, the average accuracy obtained in our study was substantially higher.

IV. RESULTS AND DISCUSSIONS
First, we averaged the recognition accuracy per participant and checked whether this accuracy correlated with the participant's hand size. Refer to Supplementary Fig. 2 for the relationship between them. We calculated Spearman's rank correlation coefficient between accuracy and hand width, height, or size. The hand size is the average of the hand width and height. The correlation coefficient between recognition accuracy and hand width was -0.01 ðp ¼ 0:97Þ, that between recognition accuracy and hand height was -0.21 ðp ¼ 0:46Þ, and that between recognition accuracy and hand size was -0.13 ðp ¼ 0:64Þ. These results suggest that there is no significant correlation between recognition accuracy and hand size. Thus, we performed the analysis without considering the participants' hand sizes.
We performed a GLM analysis to investigate how four factors of stimuli (rotational direction, starting point, end point, and number of vibrations) affected recognition accuracy. We fitted the GLM to regress the number of correct answers using these four factors. The GLM employs a logarithmic link function with a Poisson distribution. As a result of the likelihood ratio test (Type II test), there were significant main effects on the starting point [df ¼ 7; Next, we describe the results of the post hoc tests on these significant main effects and interactions. We consider that accuracy changes owing to the starting points would be caused by the error in the direction recognition of the vibration source at the starting points. Thus, we used the absolute error of the direction recognition data obtained in our previous study [9] to calculate the correlation between the recognition accuracy of the spatiotemporal patterns and the absolute error of the direction recognition. The Spearman's correlation coefficient was -0.86 ðp ¼ 0:007Þ, which showed a statistically significant correlation between them (see the relationship between them in Fig. 6(b)). This indicates that the variation in the recognition accuracy of spatiotemporal patterns due to starting points can be explained by the ability of humans to recognize direction of the vibration source at the starting points. Thus, we expect that if we are able to improve the human direction recognition ability by some methods (e.g., training), the recognition accuracy of spatiotemporal patterns will also be improved successfully.

A. Effect of Starting Point
Our results regarding the starting points are not consistent with Seo et al.'s experiment [23], in which the starting point at the ulnar side was significantly better than that at the distal, proximal, and radial sides. In contrast, patterns starting on the ulnar side were difficult in our experiments. We can attribute the inconsistency of the recognition accuracy to the device shape and degree of adhesion between skin and device. While our device is vertically and horizontally symmetric, Seo et al.' s device was asymmetric (6 cm Â 11 cm Â 1 cm). In Seo et al.'s device, the edge of the ulnar and radial side was short and the edge of the distal and proximal side was long. Since the vibration sources were spatially dense on the short side, participants in Seo et al.'s work might be able to clearly recognize the vibrotactile flow on the short side. It can explain the better accuracy with regard to patterns starting at the ulnar side than that starting at the distal and proximal side. The reason why the ulnar side was better than the radial side in Seo et al.'s work may be attributed to the adhesion between skin and device. While the participant's hand in our experiment naturally adhere to the silicone rubber sheet, Seo et al.'s participant's hand did not do so because they grip the device with a hand. It seems that the ulnar was in close contact with the device, and the radial side had a part where the object floated from the skin. Therefore, a small area of the skin received the propagated vibration on the radial side, and this may lead to low recognition accuracy of the patterns starting at the radial side. In addition to the change in accuracy due to the starting point, we considered that the variation in recognition accuracy due to the end point was also caused by the error in direction recognition of the vibration source at the end points. We calculated the correlation between the recognition accuracy of the spatiotemporal patterns and the absolute error of direction recognition. The Spearman's correlation coefficient was À0:81ðp ¼ 0:015Þ, which shows a significant correlation between them (see the relationship between them in Fig. 7  (b)). This indicates that the variation in recognition accuracy of spatiotemporal patterns due to the end points can be explained by the human direction recognition ability of the vibration source at these points. We expect that if we are able to improve human direction recognition ability by some methods (e.g., training), the recognition accuracy of spatiotemporal patterns will also be improved.

C. Effect of Interaction Between Start and End Points
As post-hoc tests of the significant interaction effect between the starting and end points, we conducted multiple comparisons between the end points at each starting point. The results are presented in Fig. 8(a). We also conducted multiple comparisons between the starting points at each end point. The results are presented in Fig. 8(b).
We found a significant difference in recognition accuracy when either the starting or end point was 0 . For instance, when the end points were at 180 , there were significant differences between stimuli starting at 0 and stimuli starting in the other three directions (90 , 180 , and 270 ). This suggests that stimuli starting or ending at 0 were difficult to recognize. In contrast, when the stimuli started or ended at 90 , it was not difficult for the stimuli to start or end at 0 . This suggests that the stimuli that started or ended at 90 were easy to recognize. Fig. 9 shows the recognition accuracy due to rotational direction of stimuli. The main effect of rotational direction was found to be significant, which suggests that the accuracy was significantly higher in the case of CW than in the case of CCW. This is consistent with a previous study [17] that indicated the recognition of CW stimuli was significantly better than the recognition of CW stimuli. In contrast, it is not consistent with previous studies reporting that rotational directions did not affect the accuracy of recognition [16], [23], [25], [39].

D. Effect of Rotational Direction
Factors causing rotational direction, which could be both physical and perceptual, are yet to be identified. To discuss these factors, let us consider the pattern ID 1 and 9 shown in Fig. 2. As physical factors, in the case of both of these patterns, the same positions 315 , 0 , 45 vibrated, and the SOAs between them are the same, but the direction of rotation is different. The direction of rotation might have a significant effect on how the vibration propagates over the skin in space and time. As a perceptual factor, there is a possibility that it is easier for humans to perceive CW and CCW rotation. Our results could be related to some participants' reports in the previous work by [25] that there was a difference in constructing mental models for CW and CCW stimuli. Still, it is unclear from our experimental results which perceptual factors made a difference.
In our experiment, all participants were right-handed. It is also unclear whether the interaction effect between the dominant hand and rotational direction was significant. Investigating the interaction effect is a topic for future research. The reason for the high accuracy when there were three vibrations could be related to the characteristics of the tactile numerosity judgment. Humans often underestimate the number of vibrations in spatiotemporal vibrational patterns [42]. Fig. 11 (a) shows the relationship between the number of vibrations of the stimuli and the answer. Fig. 11(b) shows the difference between them, that is, the number of vibrations of the stimuli minus that of the answer. In the following analysis, we confirmed that the differences between them were dependent on the number of vibrations in the patterns. Because we could not identify the distribution suitable for both positive and negative count data for the GLM, we transformed the distribution using an aligned rank transform (ART) [44]   The data for the absolute directional error of vibration localization are quoted from our previous study [9]. were performed as a post-hoc test. Fig. 11(b) presents the comparison results. It shows a significant difference between all pairs, except for the pair of 5-7. This analysis confirmed that the difference in recognition accuracy between the three vibrations and the other number of vibrations was partly due to the underestimation of the numerosity of vibrations.

E. Effect of Number of Vibrations
It should be noted that the underestimation was not strong when the number of vibrations were nine, rather than five or seven. We consider that when the participants assumed that the number of vibrations was larger than nine, they provided the answer as nine. This is because the participants only answered about the starting point, end point, and rotational direction, and it was assumed that the maximum number of vibrations was nine. Another possibility is that when the number of vibrations is nine, the start and end points are the same. If the participant noticed this similarity, it would be easy to identify the points because the stimuli were presented twice.
Since the SOA was configured differently for each number of vibrations as shown in Fig. 4, there is a possibility that participants judged the number of vibrations by the difference in the SOA. If this is correct, time and number interference [45] might have affected the accuracy. However, it is not clear from the results of this study whether or how the interference affected the result.

F. Oblique Effect
We investigated whether the oblique effect was significant. Cardinal directions are 0 , 90 , 180 , and 270 and oblique directions are 45 , 135 , 225 , and 315 . Since the number of vibrations are only odd in our patterns, when the start point is cardinal/oblique, the end point is also cardinal/oblique. Hereafter, we refer to the condition where the start or end points are cardinal (0 , 90 , 180 , and 270 ) as the cardinal condition. We refer to the condition where the start or end points are oblique (45 , 135 , 225 , and 315 ) as the oblique condition. Fig. 12 shows the examples of the cardinal and oblique conditions.     Fig. 13 shows the recognition accuracy for the two conditions. We investigated whether the recognition accuracy differed between the cardinal and oblique conditions. Here, we used non-parametric bootstrap because it does not require the specific distribution (e.g., normal distribution). The procedure of non-parametric bootstrap is to resample from the original dataset and calculate the statistical value of interest. Specifically, we calculated 10,000 bootstrap samples [46] of the differences in accuracy between the cardinal and oblique conditions. We sampled with replacement from all participants' answers and calculated the difference in recognition accuracy between the cardinal and oblique conditions. This calculation is made 10,000 times to obtain 10,000 differences. If the Bonferronicorrected 95 % CI of the difference did not overlap with zero, we could conclude that the difference was statistically significant. The calculated 95 % CI was 0.029-0.073 and the difference was shown to be significant. Based on this, we found an oblique effect in spatiotemporal pattern recognition in a twodimensional space around the hand.
We found that spatiotemporal patterns starting and ending in the oblique direction are more difficult than those starting and ending in the cardinal direction. This result is consistent with previous studies indicating that the performance of localization is better in cardinal conditions than in oblique conditions [41].
According to our results shown in Sections IV-A and IV-B, in our experimental settings, the variations in the recognition performance of the spatiotemporal pattern due to the starting or end point were significantly correlated with the absolute directional error of vibration at the start or end point. If there is an oblique effect in this absolute directional error, then the presence of oblique effect in the recognition of the spatiotemporal pattern can be explained. However, in our previous study, we did not find the oblique effect in the absolute directional error of vibration [9]. One possible cause of the difference is the difference in the number of times direction was recognized for the cardinal or oblique directions during the task. In a previous study, the participants only recognized the direction once to localize the vibration source. In contrast, in the current experiment, the participants were presented with cardinal (oblique) directions at both the start and end points because the number of vibrations was configured to be three, five, seven, or nine. This double-direction recognition might have enhanced the oblique effect, while there was no significant oblique effect for single-direction recognition.

V. GENERAL DISCUSSION
One of the purposes of this study was to investigate how well humans can recognize spatiotemporal patterns in a two-dimensional space around the hand. The probability of being correct (all of the start points, end points, and rotational direction were correct) was 48.9 %. This result can be used as a baseline when examining recognition outside the body in the future, since no studies have investigated this angle.
Another purpose of the present study was to identify which factors of spatiotemporal patterns affect the recognition accuracy. We found that the direction of the start or end points affected the recognition accuracy, which could be explained by the direction recognition error of the vibration source at these points. We also found that the rotational direction of the patterns affected the the recognition accuracy. The accuracy was slightly but significantly higher in the case of CW rotation than in the case of CCW rotation. The reason for this effect of the rotational direction of the patterns remains unclear, but the results are consistent with those of previous studies [17]. The recognition accuracy was higher when the number of vibrations in the pattern was small, which can be explained by the underestimation of tactile numerosity judgment. Furthermore, we observed an oblique effect. The patterns starting and ending in an oblique direction were more difficult to recognize than those starting and ending in the cardinal directions.
Our results indicate that the variation in recognition accuracy of spatiotemporal patterns can be partly explained by the human direction recognition ability of the vibration source at the start or end points. This suggests that improvement of the direction recognition of the vibration source by some methods (e.g., training) also improves the spatiotemporal pattern recognition. Conversely, there is a possibility that the direction recognition accuracy of vibrations within the pattern would be better than the direction recognition accuracy of a single vibration source. For example, when pattern ID: 1 in Fig. 2 is presented, the vibration sources are presented in the order of 315 , 0 , and 45 . In that case, the direction recognition accuracy for vibration from 0 might be better than single vibration from 0 that is present. It is because the humans can also use the cue of the vibrations from 45 and 315 and estimate values in the middle to recognize the direction of 0 .
Based on these findings, we identified specific patterns that were easy and difficult to recognize. The patterns that were easy to recognize were characterized by a small number of  vibrations, starting and ending in the cardinal direction but not at 0 or in a clockwise rotation. Performing a recognition task using such patterns is expected to increase recognition accuracy. When conveying symbolic information by making users discriminate between patterns, it is advisable to configure the stimuli set with such patterns. In contrast, patterns that are difficult to recognize are characterized by three or five vibrations, starting and ending in an oblique direction and counterclockwise rotation. When we wish to convey spatial information by making users recognize them, we cannot help but present users with such patterns. In such cases, it may be severe unless the low recognition rate is acceptable.
In the present study, we did not make participants select the pattern from candidate patterns. Instead, we made participants answer about the absolute pattern shape. This is because we wanted to investigate the absolute recognition accuracy. If we made participants discriminate against the patterns, the result of the discrimination would be influenced by the set of patterns. By making participants answer about the absolute pattern shape, we could clarify which factor of the patterns affects the recognition accuracy. However, from the viewpoint of symbolic information presentation, it is important to know how accurately the pattern can be discriminated from among the provided candidate patterns. In our first preliminary experiment in which all of the patterns started and ended at cardinal directions (see Supplementary Fig. 2), the IT was found to be 4.24 (see Supplementary Fig. 3). We expect that the IT would change if our 64 patterns were discriminated against because of the added patterns starting and ending in oblique direction. Like these, we believe that the results of the present study will be useful as basic knowledge for selecting a stimulus set to be used in such discrimination tasks.
In addition, we configured the repetition times of presentation of patterns so that it would be similar to previous studies [17], [23] and we can fairly compare our results with them. Since it takes more than two hours per participant in the current experimental design, it is impractical to increase the repetition times, but it is clear that the small repetition times is one of the limitations of this study.
Considering the application in the real-world scenarios, generalization of the result would be one of the future topics. The results of the experiments might be affected by the human factors or characteristics of the medium such as material or shapes. There are reports that the age [47] or the sex [48] affects the human tactile function. Also, there is a report that vibrotactile localization changed with the material of the medium in the case of a one-dimensional medium due to a change in propagated vibration characteristics [49]. Thus, to generalize the results, it is necessary to conduct an experiment with other types of materials and with people of various ages and sex as participants in future studies. In addition, in realword scenarios, there could be cases where there are distractors and users cannot focus on the tactile senses. How the result would be changed when there is a distraction is another important issue.

VI. CONCLUSION
This study investigated vibrotactile spatiotemporal pattern recognition in a two-dimensional space around the hand. Participants placed their hands on the medium and responded to the shape of the patterns. The probability of obtaining correct answers was 48.9%. This result can be used as a baseline for examining recognition outside the body in the future. In addition, we found pattern factors (e.g., start and end points or rotational direction) that affect the recognition accuracy. An oblique effect was also observed. The patterns starting or ending in the oblique direction were more difficult to recognize than those in the cardinal direction.
From the viewpoint of symbolic information presentation, it is important to know how accurately a pattern can be discriminated from the candidate patterns provided. We believe that the results of the present study will be useful as basic knowledge for selecting a stimulus set which can be used in such discrimination tasks in future studies.