Wingman-Leader Recommendation: An Empirical Study on Product Recommendation Strategy Using Two Robots

In recent years, the potential applications of social robots in retail environments have been explored. Previous studies have mainly focused on the performance of a single robot and have rarely considered the use of multiple robots to recommend products. Although a previous study did investigate the effectiveness of using two robots to recommend products, it remains unclear whether their results can be attributed a cooperative strategy or simply to the number of robots used. In this work, we explore the effectiveness of a combination strategy called wingman-leader recommendation (WLR), in which a wingman robot outside an establishment promotes a leader robot positioned inside to improve the lead robot's sales performance. Our findings suggest that the wingman robot attracted customers' attention to the leader robot and encouraged them to initiate interactions with it inside the store. We found that using two robots with the WLR strategy achieved better overall performance in terms of sales compared to using two robots without this strategy.


Wingman-Leader Recommendation: An Empirical Study on Product Recommendation Strategy Using Two Robots
Sichao Song , Jun Baba , Yuki Okafuji , Junya Nakanishi , Yuichiro Yoshikawa , Member, IEEE, and Hiroshi Ishiguro , Member, IEEE Abstract-In recent years, the potential applications of social robots in retail environments have been explored.Previous studies have mainly focused on the performance of a single robot and have rarely considered the use of multiple robots to recommend products.Although a previous study did investigate the effectiveness of using two robots to recommend products, it remains unclear whether their results can be attributed a cooperative strategy or simply to the number of robots used.In this work, we explore the effectiveness of a combination strategy called wingman-leader recommendation (WLR), in which a wingman robot outside an establishment promotes a leader robot positioned inside to improve the lead robot's sales performance.Our findings suggest that the wingman robot attracted customers' attention to the leader robot and encouraged them to initiate interactions with it inside the store.We found that using two robots with the WLR strategy achieved better overall performance in terms of sales compared to using two robots without this strategy.
Index Terms-Service robotics, design and human factors, social HRI.

I. INTRODUCTION
S OCIAL robots have increasingly been discussed in recent decades.In contrast to industrial robots used in factories or logistics centers, social robots are designed to interact with people and are generally capable of communicating with a variety of modalities such as natural language [1], [2], facial expressions [3], [4], and gestures [5], [6].This allows them to be used in many everyday environments, including to provide information [7], [8], in education [9], [10], [11], in healthcare [12], [13], and in domestic settings [14], [15], [16].Among their many applications, social robots are considered promising for the service industry.At present, robots are still novel in public places, and interacting with robots could be considered fun and enjoyable or relaxing [17].Therefore, researchers and service providers have begun to explore the use of social robots to attract customers' attention [18], [19], provide information [20], and improve customers' experiences [21].The use of social robots to make recommendations has recently attracted attention [18], [22], [23], [24], [25], [26], [27].As a persuasive technology, social robots could provide a promising approach to influence people's behavior.
Previous studies have often investigated the effectiveness of a single robot; however, the potential use of multiple robots to make recommendations remains unclear.Compared to a single robot, the use of multiple robots has the advantage of greater social influence.For example, Shiomi et al. [28] explored the persuasive influence of multiple robots operating together compared to a single robot.They found that participants conformed more to the influence of multiple robots owing to an increased sense of pressure.In another study, Kamei et al. [29] proposed the use of three communication robots in a retail sales environment and allowed them to recommend items to customers.Their results indicated that the robots influenced customers' behavior effectively.However, previous works did not consider evaluating specific strategy for the use of multiple robots, which could limit the practical application of groups of social robots in sales.
In retail, placements inside and outside of an establishment are typically considered for applications with social robots [18], [27].Considering a typical customer flow that starts before entering a store, whether a robot is placed outside the store as the first contact point could influence a potential customer's mental state and behavior.Positioning a robot outside a store could attract the interest and attention of potential customers and further prepare them for subsequent contacts with another robot inside the establishment.Song et al. [30] explored the effectiveness of using two robots together with one inside and another outside to improve performance on a product recommendation task.Specifically, they used one robot inside a store to recommend products and another outside the store to promote the robot inside and introduce products to customers.Their results showed that using two robots was more effective compared to using only a single robot inside.However, as the authors discussed, the robot located outside was effective in introducing the recommended products to customers, which naturally increased the number of times the products were mentioned to the customers.As a result, whether their findings could be attributed to the strategy implemented in using two robots together or simply to the greater number of robots remains unclear, given that using more robots increased the number of times the particular products were mentioned to customers and presumably made a stronger impression in terms of mere exposure [31].Hence, we need to further investigate and better understand the effectiveness of combination strategies that use multiple robots for product recommendation.
We propose a recommendation strategy, called "wingmanleader recommendation (WLR)", which is designed for situations in which a customer encounters two robots one after the other.Based on [30], in this letter we refined and formally proposed WLR as a new collaborative recommendation strategy using multiple robots.The WLR strategy includes a wingman robot (e.g., positioned outside) and a leader robot (e.g., positioned inside).The leader robot recommends products to potential customers, while the wingman is primarily tasked with promoting the leader robot.Specifically, we assume the two robots are located separately and cannot be viewed simultaneously to ensure the customer's curiosity about the leader robot when he or she interacts with the wingman robot.In this study, we investigated the effectiveness of using the WLR strategy by comparing the results of experimental conditions with and without WLR.We conducted an experimental evaluation at a souvenir store in Kobe Airport in Japan.Our results indicate that the use of a WLR strategy achieved better performance overall compared to the use of two robots without any strategy applied.We used the Wizard of Oz (WoZ) as a method to explore the design of interactions.When further designing automated robot systems for product recommendation, we suggest better consider collaborative interaction strategies using multiple robots.These findings can contribute to the effective application of social robots in retail sales.

II. METHDOLOGY
This experiment was approved by the Research Ethics Committee of Osaka University (Reference number: R1-5-9).

A. Robot System
We used Sota robot in this study.Sota is 28 cm tall and has a child-like morphology.We developed a robot system to implement the Wizard of Oz (WoZ) [32], [33] methodology.Specifically, the system comprises two main parts, including a robot controller and an operator interface (Fig. 1).A control box was placed under a Sota robot, and the controller program was run on a mini-PC hidden inside the box.The controller manages the motion and speech of the robot and communicates with the operator interface via the Internet.An ultra-wide-angle webcam was installed behind the Sota robot to capture video from its perspective, and the video stream was sent to the operator in real time.In addition, the system was configured to generate commands based on the data received from the operator and send them to Sota to control their behavior.

B. Wizard of Oz
The WoZ technique has been adopted in the research on human-robot interaction (HRI) for some time owing to its effectiveness in exploring design ideas and observing user behavior without excessive development cost during the prototyping phase [34].In this study, we adopted the WoZ technique to overcome technological limitations such as those of natural language conversation and speech recognition in noisy environments.In addition, studies on HRI in uncontrolled environments often involve various and complex situations [32].The use of WoZ enabled the robot operators to handle various practical and unexpected situations and tasks that were not specifically planned for during the design phase.Therefore, we considered the adoption of the WoZ approach as a necessity to achieve the objectives of this work.
For the experiment, we employed female voice actors to operate the robots with the WoZ approach.All the operators were native Japanese speakers in their 20 s, and were highly skilled performers and communicators qualified to engage in scripted conversations (e.g., product recommendation) and improvised conversational interactions (e.g., chitchat).In the experiments, we presented the robots to consumers as childlike autonomous conversational systems, and the operators were asked to stay in character as a robot.For example, when chatting about traveling, an operator would avoid saying "I have travelled to [some] place," and instead would use phases such as "the place seems nice" because the robot would not be able to travel; the operators were also asked to avoid coughing or laughing without muting their microphone to maintain the illusion of the robot being an intelligent computational system.Additionally, the pitch of the operators' voices was shifted higher to match the childlike appearance of the Sota robots during the experiments.

C. Experiment 1) Field:
We conducted an experiment in a souvenir store called "Kansai Tabi Nikki" located in Kobe Airport in Japan.The store is located in the departure lobby, and many travelers visited to shop for souvenirs before the security check.The store carries a diverse selection of various souvenirs, and especially sells snacks and sweets as well as food and drink from the Kansai area.Fig. 2 shows the layout of the store and the experimental setup.
2) Experimental Design: As in [30], we considered two locations for the robots, including a position outside the store at the entrance and a position inside the store in the back.Based on these placements, we considered the following three conditions in the evaluation from the perspective of the promotion of the inside robot.
r No Promotion (NP): only the inside robot was employed.r Implicit Promotion (IP): both inside and outside robots were employed; the two robots simply recommended products on their own without using any particular strategy to work together.
r Explicit Promotion (EP): robots were positioned both in- side and outside the establishment; the WLR strategy was adopted with the robot positioned outside assigned as the "wingman" to explicitly promote the "leader" robot inside; the outside robot would additionally introduce the products if a customer continued talking with the robot and asked it for recommendations.The robot inside the store was primarily used to recommend products.We discussed this with the management team of the souvenir store and decided to recommend eight kinds of souvenirs with prices ranging from approximately 600 to 2,200 Japanese yen.The robot operators used the basic strategy of recommending products based on customers' needs.For example, the robot could ask a customer for whom they were shopping and make suggestions for souvenirs to purchase based on their answers.The suggestions for souvenirs for different needs were provided by the store before the experiment. 1he tasks of the robot placed outside the store depended on the experimental conditions.Under the IP condition, the robot located outside the store performed the same tasks as one inside to recommend products to shoppers.Importantly, the robot outside the store avoided mentioning the presence of the other robot inside.In particular, we asked the operators in the IP condition to avoid mentioning one another to the customers.In the EP condition, the outside robot was assigned the role of "wingman" and the operator was asked to explicitly promote the inside robot acting as a "leader".As an example, a typical script for promoting the robot in the shop could be given as "My brother (name of the robot inside) is inside the store and he knows a lot about the souvenirs.Please go visit him and ask him to recommend the perfect souvenirs for you!"In addition, the robot would welcome the customers and provide suggestions on the products if a customer continued talking with the robot and asked it for recommendations.
For the IP condition, we considered the presence of the robot outside to have an implicit promotional effect on the unit inside the store.Specifically, the use of a robot positioned outside an establishment as the first contact point for a potential customer could attract their interest and attention and help them to adapt to the particular situation of the robots in service.In addition, the use of robots both outside and inside for recommendation could increase the number of times particular products were mentioned to customers.According to the mere-exposure effect [31], customers can form stronger impressions of a product if they are exposed to it repeatedly.Therefore, a robot outside a retail store may affect the recommendation performance of a robot inside regardless of whether a combination strategy is adopted.However, because this promotion is implicit, the influence of the robot outside could be weaker than that of the EP condition, in which the robot outside verbally advised shoppers as to the presence of the other unit inside and its capability to recommend suitable products.
The experiment was performed for 12 days over a period of 2 weeks.Each experimental condition lasted for about four days and was included operating hours on both weekdays and weekends.The experimental period lasted for 5 hours per day Fig. 3. Effect of robot which was calculated based on the predicted value of the sales.from 2 pm to 7pm, which were the peak hours for visitors.The same days of the week were used for the three conditions to measure the changes in sales.
3) Evaluation: We relied primarily on evaluations of sales and customer behavior toward both inside-and outside-store robots.We obtained sales numbers for all products from the souvenir store, including both recommended and the other products.Based on the data, we calculated the increase in the sales of the recommended products as well as the total sales of the store by comparing the sales data between the base and experimental periods.This allowed us to track increases in the sales of recommended products as well as to obtain some interesting data such as how the robots' recommendations affected the sales of souvenirs that were not recommended by the robots.
During the experiment, due to a typhoon2 and holidays,3 we observed a dramatic decrease in the number of customers visiting the store for approximately half the days during which the experiment was conducted days.It was difficult to temporarily adjust the schedule of the experiment due to unexpected factors such as typhoons, as coordination with the field side had already been done several months before the experiment.Therefore, we did not directly compare the raw sales data across different experimental conditions to evaluate the effects more appropriately.Instead, we sought assistance from a potential outcome framework [35] and calculated the predicted value of the sales.Specifically, we obtained both past sales data and that recorded during the experiment.Based on the experimental period, we were able to divide the sales data into off-and on-hours.Off-hours were the periods without a robot on duty, from 6am to 1pm and from 7pm to 9pm.As a result, we obtained four parts of the sales data, including 1) baseline on-hours S b_on , 2) baseline off-hours S b_of f , 3) experiment on-hour S e_on , and 4) experiment off-hour S e_of f .Furthermore, we were able to calculate the predicted value of sales for the experiment during on-hours SP e_on using (1) as given below.(Fig. 3) In general, SP e_on predicted the sales during the period of the experiment without any robots employed.We assumed that the ratio of S e_of f SP e_on in the experimental period would be the same as the ratio of S b_of f S b_on during the baseline period.However, the actual sales S e_on differed from SP e_on , and we attribute this difference to the effects of the robots.Consequently, we were able to calculate the effects of the robots on sales using (2).(Fig. 3) We used E robot to evaluate the effects of each experimental condition.
In addition, we recorded the experiment on video using webcams on the robotic system for behavior annotation.In particular, we focused on customers' behavior toward the robots because we wanted to observe how they behaved differently in each experimental condition.In the evaluation, we used the chi-square test to identify differences in customers' behavior such as whether they talked about the recommended products with the robot or initiated conversational interactions.

A. Sales
Table I lists the sales of the products across all three experimental conditions.For the NP condition, in which only a single robot inside was used for product recommendation, no effect of robot on sales was confirmed.Sales of all souvenirs did not change.Specifically, sales change of the recommended souvenir items were E robot = −10 thousand yen while sales change for the other items were E robot = 10 thousand yen.For IP condition, in which both robots simply recommended souvenirs on their own without collaborating, larger changes in sales were observed compared to NP.Although sales for all souvenirs decreased by E robot = −10 thousand yen, the use of robots lead to strong product recommendation effects, with sales increased of E robot = 70 thousand yen.Interestingly, the sales of other souvenir items showed an opposite change, decreasing by E robot = −80 thousand yen, which contributed to a total decrease in sales for all souvenirs.For EP condition in which the outside robot was assigned the role of "wingman" to explicitly promote the "leader" robot inside, the results showed overall the most promising product recommendation effects.There was a significant increase in sales of E robot = 120 thousand yen for all souvenirs.Furthermore, there was both a relatively large increase of E robot = 100 thousand yen for recommended souvenirs and an increase of E robot = 20 thousand yen in sales of other items.

B. Customer Behavior
For the robot outside, we focused on a comparison between the IP and EP because no robot was placed outside the store as a wingman robot in the NP condition.Table II lists the annotation results for the customers' behavior.To be specific, the ratio of customers who talked to the robot was 63.3% for IP and 64.2% for EP, and no significant difference was observed.However, the ratios of customers who talked to the robot about the items were 42.9% for IP and 28.4% for EP.The results of a chi-square test indicated significant differences, with χ 2 (1) = 6.24, p < 0.05.The percentages of customers who showed interest in the recommended souvenirs were 13.6% for IP and 5.5% for EP.The results of the chi-square test indicated significant differences with χ 2 (1) = 4.73, p < 0.05.In addition, 47.5 % of the customers entered the store in the IP condition and 69.7% in the EP condition, and the results of a chi-square test showed significant differences, with χ 2 (1) = 13.83,p < 0.05.
For the robot inside, we focused on comparing the NP, IP, and EP conditions.Specifically, the proportions of customers who talked to the robot were 49.6% for NP, 47.8% for IP, and 46.4% for EP.No significant differences were observed.The percentage of customers who talked to the robot about items for sale was 56.1% for NP, 57.0% for IP, and 50.8% for EP; no significant difference was found.There were no significant differences in the percentage of customers who showed interest in the recommended items 11.9% for NP, 10.4% for IP, and 11.7% for EP.However, 8.9% of customers initiated conversations with the robot in the NP conditions, 10.9% did so in the IP condition, and 20.5% did so in the EP condition.The results of a chi-square test indicates a significant difference with χ 2 (2) = 27.67,p < 0.05.The residual analysis in comparison with the mean across all conditions indicated that the NP condition was associated with a significantly lower ratio (r = -3.34,p< 0.01) compared to the EP condition (r = 5.19, p < 0.001).A post-hoc binomial test confirmed that the EP condition had a significant higher ratio compared to both NP (p < 0.001) and IP (p < 0.001) conditions.

A. Sales of Souvenir
Our results showed the effectiveness of using two robots to recommend products compared to using a single robot.As shown in Table I, the employment of robots in the NP condition had no effect on sales overall.This replicates the results in [30] in that there was no significant increase in the condition with a robot inside only.However, both IP and EP conditions in which two robots were employed, achieved a relatively large increase in the sales of recommended souvenirs (E robot = 70, 126% increase in IP and E robot = 100, 132% increase in EP).This shows a similar result in [30] as well, in that using the outside and inside robots achieved a significant sales percentage of the recommended items compared to other conditions.Therefore, we demonstrated that a higher performance in product recommendation could be achieved using two robots compared to a single robot.
Moreover, although positive effects on the sales of recommended items were observed in both the IP and EP conditions, we found an interesting results that better performance was achieved in EP compared to IP in terms of overall product recommendation effectiveness.To be specific, although a positive increase in the sales of recommended souvenirs was achieved in the IP condition (E robot = 70, 126%), this unfortunately was associated with a large decrease in sales of items other than those recommended (E robot = -80, 90%).Consequently, the overall sales of all souvenirs decreased slightly (E robot = -10, 99%).On the other hand, an increase in sales was achieved for all items in the EP condition, including both the recommended items (E robot = 100, 132%) and other products (E robot = 20, 102%), which contributed to an overall increase in the sales of all items (E robot = 120, 111%).Thus, these results show that the EP condition in which the wingman robot explicitly promoted the inside leader robot, was the most effective for product recommendations, and contributed to the overall sales of the store.Plausible reasons for this are discussed below.

B. Power of Wingman-Leader Strategy
As summarized in Table II, we annotated the video of customers' behavior toward the robots to explore what factors potentially may have impacted differences in the effects of the robots on sales in each condition.The results reflected the features of each condition.The main task of the robot located outside the store in IP was to introduce and recommend souvenirs, whereas its main task in EP was to promote the lead robot inside the store rather than to recommend products directly; therefore, we observed that significantly more customers talked to the robot outside about the items and showed interest in the products on offer in IP compared to EP.However, because the promotion strategy used in EP invited customers to enter the store and talk to the robot inside, it could attract customers' interest in the robot inside and encourage them further to enter the store.Therefore, the results show that significantly more customers entered the store in the EP compared to IP.
For the robot inside the store, we compared the customer behavior under all three conditions.Although the results showed no significant differences with regard to customer behaviors of talking to the robot, talking to the robot about items, and showing interest in items, an interesting result was observed with regard to customers initiating conversations with the robot.Specifically, a significantly higher ratio of customers who initiated conversations with the inside robot was observed in the EP condition compared with both the NP and IP conditions.This could be due to the promotion strategy of the wingman robot.A customer who influenced by the promotional information could become interested in the seeing the robot inside the store and therefore enter and seek out the robot to talk to it.
Interestingly, as shown in Table I, although sales of recommended souvenirs increased in both IP and EP conditions, we recorded a large decrease in sales of items other than those recommended, which led to an overall decrease in sales of all souvenirs.In contrast, the EP condition showed an increased in sales of all items.One plausible reason could be that the two robots in IP simply recommended products on their own without collaborating.Customers may have been primarily influenced by information on recommended souvenirs and fail to consider other products.Consequently, customers ignored the items that were not recommended, whether consciously or unconsciously, and fewer sales were recorded.However, the wingman-leader strategy used in EP eliminated this issue.Customers developed an interest in the leader robot and proactively talked to it.They remained consistent in their interest in the robot and credited its recommendations, which did not seem to negatively influence their intent to purchase other souvenirs.This shows the effectiveness of the proposed wingman-leader recommendation strategy.

C. Design Implications
Based on [30] and the results of this study, we have confirmed the effectiveness of using two robots together to recommend products.We demonstrated that the use of a combination strategy for two robots, in which one robot promotes another, can be effective for achieving good performance.In this study, we showed that simply using two robots and allowing them to recommend products on their own without collaborating failed to achieve an overall satisfactory performance, whereas our proposed wingman-leader strategy, which includes two robots with different roles such that one wingman robots promotes the leader robot successfully improved the effectiveness of the leader robot and achieved better performance overall.We used the WoZ as a method to explore and evaluate the design of interactions.When further designing automated robot systems for product recommendation, we need to better consider collaborative interaction strategies using multiple robots.Recent advancements in Large Multimodal Models (LMMs), which integrate additional modalities into Large Language Models (LLMs), may facilitate the development of robotic automation for these interactions.For instance, incorporating computer vision into LMMs could enable robots to better comprehend customer behaviors, like expressing interest in or inspecting a product.We believe this enhancement could substantially elevate the interaction experience and efficiency.Additionally, previous work [36] has shown that retailers expect robots to welcome customers and recommend products, and product recommendation is considered an important task for such robots; [37] has claimed that more empirical research is needed on the actual behaviors of service robots.Our empirical findings offer insights and implications for the design of compelling and effective applications of service robots.
However, it's important to note that the use of service robots for product recommendations could potentially introduce negative societal influences.For instance, retailers might leverage this technology to prompt impulsive and unnecessary purchases of expensive items.While, currently, we are unable to provide specific recommendations on how to prevent this, we suggest that attention to the ethical concerns regarding the use of service robots for product recommendation is crucial.This responsibility extends not only to robotics researchers and developers but also to retailers and management.

D. Limitations and Future Work
There are several limitations in this research that could be addressed in the future.First, we used the Wizard of Oz (WoZ) method to investigate the design of interactions involving multiple robots.Future studies need to explore the design and development of interactions for automated robot systems.Second, this letter proposed the WLR as an effective product recommendation strategy that involves the collaboration of two robots.We suggest future research explore diverse collaborative methods, robot placements, and the potential for using more than two robots.Third, it was unfortunate that, due to unexpected factors such as a typhoon, we were unable to directly compare the sales data across the experimental conditions.Instead, we utilized a potential outcome framework to calculate the predicted value of sales for evaluation.While we believe this method is valid, it's important for readers to note this limitation in our analysis.In addition, we suggest that future studies conduct field experiments in a variety of facilities to garner more universally applicable insights for successful real-world applications.

Fig. 2 .
Fig. 2. Layout of the souvenir store and robot settings for the experiment.