Comparing visual representations of collaborative map interfaces for immersive virtual environments

Virtual reality offers unique benefits to support remote collaboration. However, the way of representing the scenario and interacting within the team can influence the effectiveness of a collaborative task. In this context, this research explores the benefits and limitations of two different visual representations of the collaboration space, shared experience and shared workspace, in the specific case of map-based collaboration. Shared experience aims at reproducing face-to-face collaboration in a realistic way whilst shared workspace translates to the virtual world the functionalities of 2D collaborative spaces. The goal is to understand whether sophisticated interfaces with realistic avatars are necessary, or if simpler solutions might be enough to support efficient collaboration.We performed a user study (n = 24, 12 pairs) through a collaborative task with two roles in a emergency crisis intervention scenario that typically uses map-based interfaces. Despite that a shared experience scenario might provide a better personal experience to the user in terms of realism, our study provides insights that suggest that a shared workspace could be a more effective way to represent the scenario and improve the collaboration.


I. INTRODUCTION
T HIS paper explores the effects of visual scenario representation in a collaborative map interface for immersive virtual environments. A possible use case of a map interface for remote collaboration could be virtual crisis rooms. In crisis rooms, experts from different fields meet to discuss and propose solutions for an ongoing situation. These rooms are usually equipped with a big wall-screen in which the current information about a crisis is displayed, and the experts use it to discuss and analyze the data. Previous works of the research group focused on the use of multi-device environments integrating different devices for collaborative information analysis such as tabletops, vertical displays, desktop computers or tablets [8]. However, these types of environments require very specialized and high cost equipment not always available.
The use of immersive technologies could be a suitable alternative. Firstly, equipment with similar functionalities as the physical ones can be simulated in virtual environments reducing the costs needed for implementing the technology [38]. Secondly, the implementation of virtual spaces has the potential to facilitate remote collaboration almost as similarly as in face-to-face settings. Immersive technologies are expected to help to overcome some of the drawbacks of traditional 2D displays for collaboration since they can provide natural interaction, engagement, and infinite workspaces [12].
However, the use of virtual environments to support collaboration faces challenges that need to be carefully considered. First, it is necessary to provide effective ways of communication and to establish engagement between collaborators [2]. Second, these kinds of collaborative environments require to effectively provide workspace awareness [18]. In face-to-face environments, it is relatively easy to gather information about the state, intentions, and actions of the rest of the collaborators. However, in computer mediated collaborative environments part of this information might be missed, which makes it necessary to provide effective mechanisms for supporting such workspace awareness. This might seem easier to achieve in virtual environments where realistic avatars can be represented. But if immersive technologies have to reach a broad audience, simpler and more affordable solutions might be enough to support efficient collaboration. Indeed, too realistic metaphors for visual interfaces might be cumbersome and clipper the interface with irrelevant information [14].
This work explores the benefits and limitations of two different scenarios for visually representing workspace awareness cues in a collaborative map interface for immersive virtual environments: shared experience and shared workspace. In a shared experience scenario, workspace awareness cues are presented in a way that reproduces face-to-face settings. In a shared workspace scenario, workspace awareness cues are presented in a similar way as traditional 2D-computermediated collaboration. In particular, we will focus on studying the visual representation of virtual spaces, letting aside other multisensory feedback, and on distributed synchronous collaboration scenarios [22].
We conducted a user study to measure efficiency and workload of users when using the two different visual representations. Intuitively, a realistic representation of the scenario like a map interface situated in a crisis room could perform better because it is more natural and familiar to the users. However, the results of the study with 24 participants suggest that the collaboration is more efficient and requires less workload levels in a shared workspace scenario. Furthermore, the participants of the study did not report greater differences in terms of social presence between both scenarios.
The rest of the paper is organized as follows. Section 2 presents some related works. Section 3 presents the methods of the study. Section 4 describes the collaborative map interface prototype. Section 5 presents the description of the experiment. Section 6 describes the data collection of the study. Section 7 presents the results and analysis of the study. Section 8 discusses the results of the study, and presents its limitations. Finally, in the last section, we present the conclusions and future works.

II. RELATED WORK
Early areas of research in immersive CSCW focus on understanding the role of collaborative behaviors in physical spaces. Ens et al. [13] identified two theoretical issues that emerge in this area: (1) understanding how to create awareness of collaborators, that is, generating knowledge of who is in the workspace and what are they doing; (2) understanding how to support collaboration through visual information. The rest of this section will review these issues. The first part covers basic concepts and related work of workspace awareness when using immersive VR technology. In the second part we will center on the visual representation of the collaborative spaces. Finally, the third part summarizes the works reviewed and discusses their implications for our study.

A. WORKSPACE AWARENESS
Workspace awareness is a relevant factor in CSCW as it gives users information about what is happening in a collaborative space, and how they can contribute to succeed in the collaborative task [18]. Gutwing and Greenberg propose a threepart framework [19] to know what information constructs workspace awareness, how to gather it, and how to use it in collaboration. Focusing on synchronous collaboration, three categories of workspace awareness are identified: awareness of presence, awareness of actions, and awareness of location. Awareness of presence intends to gather information about who is in the workspace. Awareness of actions intends to gather information about what the other people in the workspace are doing. Awareness of location intends to gather information about the location, reach area, and field of view of the other people in the workspace.

1) Awareness cues for social presence (Who)
Also known as co-presence, social presence refers to the "sense of being with another" [3]. Social presence is one of the three dimensions of presence (spatial presence, selfpresence, and social presence). Weinel et al. [49] identify social presence as a relevant factor in collaboration. In their study, they identify that social presence positively influences the perception of the task workload and collaboration. Furthermore, Roberts et al. [41] investigate the influence of social presence and group size in group interaction. In their study, they compare groups of two different sizes in three different settings: face-to-face, face-to-face using collaborative software, and virtual using collaborative software. The results of their study suggests that the use of collaborative software enhances the feeling of social presence, reducing the negative impact derived from increasing the size group.
In the specific case of the collaboration supported by immersive VR technologies, social presence plays even a more decisive role. The results of the study presented in [35] suggest that social presence has a tight relationship with social influence, and that improves communication, coordination, and trust between collaborators.
Several studies reported that VR collaborative environments that use avatars to visually represent other collaborators produce higher levels of social presence than those that do not use them [24]. The sense of presence can be enhanced when those avatars represent realistically the current actions and behaviors of the collaborators. For example, the results of the study presented in [48] suggest that participants felt a stronger social presence when they interacted with agents that displayed appropriate feedback behavior by nodding its head in comparison to those who did not. Similarly, the study of Pan et al. [37] showed that participants felt higher levels of social presence when a virtual agent blushed after making a mistake during a presentation.
However, it is possible to find studies which call into question the relationship between the use of avatars and the increment of social presence. Robb et al. [40] performed a user study to investigate how a human trainee's presence affects behavior of nurses and surgical technicians during virtual team training. The results of the study showed no statistical differences in the training outcomes. However, they identified some notable differences when humans interact with virtual teammates in other aspects such as predisposition to assists and efficiency of task execution.
2) Awareness cues for actions (What) Previous works in this area focus on tools to support mutual understanding of the work done by the collaborators. Most of these tools can be classified into two categories: visual-based tools, and non-visual-based tools. In this section we will focus on the mutual understanding of communication acts. These are specially relevant to our work, as they allow the collaborators to communicate their intentions or instruction to the other.
Visual-based tools provide a low cost solution to gather information about the actions performed in collaborative spaces. A simple way to represent intentions, emotions, and social communications in general is the use of emojis. This image representations can substitute words in a speech [44], express actions [42] or even arise emotional awareness among the collaborators [4]. However, emojis can also reduce the perception of competence [17] specially in formal settings, or result in ambiguous understanding of the message [44].
Non-visual-based tools would include those systems that provide information about the collaborators' actions using auditory cues. The most straight way to convey communicative acts among the collaborative space is the use of the voice. This channel is preferred by the users when solving ambiguities of specific gestures like deictic gestures or conflicts during collaboration [45]. However, voice is also considered an intrusive way of communication, or it can be interpreted as a demand to immediate action [16].

3) Awareness cues for location (Where)
Location awareness cues refers to those systems for providing users with information about the whereabouts of their collaborators, the area they are looking at, or the artifacts they can reach at a given moment, for example. In VR, the most common ways to present these cues are pointing gestures, ray pointers, viewports, and frustums.
Pointing gestures and ray pointers work in a similar way, allowing a collaborator to provide indications by pointing at a target in the virtual space. Ray pointers represent the path from the pointing gesture to the target, usually with a straight line, and are particularly useful in virtual environments where the pointing gesture might not be precise enough to identify the target. Li et al. [31] propose the use of a ray to represent the collaborator gaze in an AR application for remote expert help when navigation across a campus. On the other hand, pointing gestures use solely the expression of the collaborator, and the user has to estimate the place that the collaborator is referring to. Works like [45,33] focus on improving the perception of the user to facilitate the understanding of pointing gestures.
Viewports employ a frame to present the user the current area of view of the collaborator. Although they are traditionally used in non immersive collaborative tools as 2D displays, they have also been successfully implemented in 3D environments [28]. However, in these types of settings "frustums" are more commonly used. In this case, the 3D area the collaborator is viewing is demarcated within the edges of a prism. The results of the study presented by Piumsomboon et al. [39] suggest that the use of frustums improves the user performance.

B. REPRESENTATION OF THE COLLABORATIVE SPACE
Ens et al. [13] propose a matrix with six dimensions to categorize the work developed around collaboration on MR. The first two dimensions are the classic CSCW matrix proposed by Johansen [23], and the other dimensions are related to the technologies and characteristics inherent to MR.
The scenario dimension of the matrix takes special relevance to our research as it refers to the different possibilities to represent the collaborative space and the awareness cues, in particular the shared workspace and the shared experience. Shared workspace are the systems or studies that have a strong focus on the workspace. In the case of shared experience, the representation of the space does not only focus on the task that the collaborators are working on and its workspace, but also on the personal experience of the collaborators. In the next sections, we summarize some examples of environments that follow these two approaches.

1) Shared workspace
Some examples of use of this type of scenarios are: physical games like [36,25], construction discussion [32], or board games [46]. All of these examples have in common that the focus of the collaboration is on the task or the workspace, rather than the feel of co-presence or the details in the scenario.
Yasojima et al. [51] propose a collaborative AR tool for information visualization support. Their tool makes use of markers to display a 3D visualization of the real world, and share the visualization collaboratively. In this case, the visualization tool implements a shared workspace scenario as the focus of the tool is the visualization, and the rest of the elements of the scenario or the personal experience of the collaborators are not supported by the AR tool.

2) Shared experience
Some examples of these type of scenarios are: disaster rescues [34] where a remote expert require a detailed visualization of the rescuer surroundings to provide them help, or museum explorations [6] where the guide and the tourists need to be aware of the presence of the other person to understand its explanation and a detailed view of the elements of the scene.

VOLUME 4, 2016
Barden et al. [1] explore the opportunities that AR technologies could provide to enhance telematic events like a dinner party. In their work, they explore the use of some visual cues to support remote guests to experience togetherness and playfulness. This work is a good example of shared experience scenarios where the guests of the dinner party need to have a detailed view of the actions, facial expressions, or activities of the other people present in the dinner table to know if the other guests are having fun or feeling awkward, for example.
Another example of shared experience are the metaverses. This novel concept illustrates the interaction between humans in virtual spaces [9]. In the metaverses conceived by Facebook 1 or Microsoft 2 , the personal experience of the users require a detailed representation of the space, realistic interaction between the users, and natural interfaces to perform tasks in a virtual space.

C. MAP INTERFACES IN VIRTUAL REALITY
Previous works [50] have investigated the outcomes of different types of representations of maps in VR spaces, as exocentric globes, flat maps, egocentric globes, and curved maps. Their results suggest that exocentric globes perform better for distance estimation. However, flat maps allow to display all the map surface at once, in a way the users are familiar with. Further research has explored the use of authoring tools to integrate flexible visualizations of maps in virtual environments [30].
With regards to the interaction style with the map interface, our previous studies suggest that hand controllers, such as the Oculus Touch, provide a more efficient and usable mechanism for controlling the map than gesture-based solutions [43]. Other researchers have explored the use of novel interaction methods for map interfaces like proxemic interaction [15].
For achieving common ground in map interfaces, Convertino et al. [7] identified that communication between the collaborators can be improved through specific tools for sharing awareness like viewports or role indicators. These tools enhance knowledge-sharing and activity awareness.

D. SUMMARY OF RELATED WORKS
In this section we presented previous work done for workspace awareness and visual representation of collaborative virtual spaces. Several tools and solutions have been presented to provide awareness cues of the presence, action, and location of other users in a collaborative space. However, the specific area of map interfaces for VR is still underinvestigated, and there is a need to research which visual representation works better for virtual collaborative spaces. For this case, we identified two possible representations: shared experience and shared workspace. Despite both scenarios have positive and negative qualities, there is a lack of studies that provide insights of when to use them and how to effectively use them in collaborative immersive scenarios. This study tries to cover this gap and contributes on how to effectively represent workspace awareness in map interfaces for virtual environments.

III. ANALYZING VISUAL REPRESENTATIONS OF IMMERSIVE MAPS INTERFACES FOR COLLABORATIVE SPACES
The purpose of this study is to gain understanding about how the visual representation of the scenario (shared workspace and shared experience) in immersive map interfaces affects the collaborative task. To achieve this objective, we set the following research question: RQ Which is the most suitable visual representation of the scenario for collaboration in an immersive virtual reality collaborative map interface? It is expected that the shared experience representation will be more intuitive as it reproduces the way the users work in the real world. For example, users could use pointers to mark locations in the map, as they would do in a real crisis room. Also, they will perceive the presence of other team members through their avatars, as in the real world. However, replicating a real scenario makes it necessary to represent additional elements, as pieces of furniture, that are not directly related with the task to perform. Moreover, the size available to display the map will be limited, as it would be in a real room.
On the contrary, in the shared workspace scenario the entire field of view of the user can be used to display the map, in a similar way as in a traditional 2D map interfaces. However, the lack of certain social cues might diminish the user experience and the feeling of co-presence. This could negatively impact the user perception of workload.
Based on this rationale, we present the following hypothesis: H1: Users will collaborate more efficiently in a shared experience scenario in comparison to a shared workspace scenario. H2: Users will perceive lower workload levels in a shared experience scenario in comparison to a shared workspace scenario. As discussed in the previous section, the social presence in a collaborative system is usually regarded as a relevant factor to enhance collaboration. For this reason, in this study we will also analyze the perception of the user about their social presence.

IV. THE COLLABORATIVE MAP INTERFACE PROTOTYPE
To compare the outcomes of these two ways of representing the virtual collaborative environment (shared experience and shared workspace), we implemented two collaborative map interfaces following each of the two approaches. In this section we present the rationale that guided the design of the awareness cues for the two environments, and their final implementation.

A. DESIGN PROCESS OF THE AWARENESS CUES 1) Identification of the awareness cues
The first step was to select a set of awareness cues to facilitate the collaboration among the users. These in terms of the three parts of the workspace awareness framework: social presence, actions, and location. Table 1 presents the identified awareness cues, summarizing the benefits and limitations identified in the literature for each cue.

2) Online survey
In the second step of the design, we ran an online survey with 37 responses to identify users' preferences for representing the communication acts with hand gestures and emojis.
The online survey asked participants to identify the gesture or emoji 3 best represent the communication acts of showing approval, showing negation, showing neutrality, asking for help, among others.
The result of the survey indicated that participants had the best agreement for the representation acts of showing approval, showing negation, and asking for help. Furthermore, the survey identified that the "index pointing up" emoji was best for asking for help (34.3% of agreement), and the "thumbs up" emoji was best for showing approval (74.3% of agreement).

3) Pilot test
The main purpose of the pilot test of the prototype was to test all the awareness cues in each scenario. Consequently, the third step of the design of the prototype was to run a pilot test with 2 couples (4 participants) following the same protocol of the main experiment explained in Table 4. Below, we present and discuss the main findings related to the use of the cues in each scenario: • Avatar and hand gestures: The participants reported that the use of avatars in the shared workspace scenario disturbed their task as they occluded the map. • Pictorial cues for communication (emojis): The subjects of the experiment felt familiar with the emojis and used it as intended in the shared workspace scenario. However, in the shared experience scenario, participants preferred to express the communications acts through hand gestures. • Auditory cues (voice): Some participants tended to use only this channel to communicate, as they considered it was the most straightforward mean to reach the collaborator. However, sometimes they felt that listening to the voice of the collaborator was distracting and hindered their work. • Pointer/Ray and viewport: Pointers were effective for indicating a location in the map to the other collaborator. However, as sometimes participants were looking at

B. AWARENESS CUES IMPLEMENTED IN EACH SCENARIO REPRESENTATION OF THE MAP INTERFACE
Considering the results of the pilot test and the literature review we designed the following awareness cues for each scenario representation .

1) Awareness cues for social presence
For the case of social presence awareness we provide the following cues: • Cues for the shared experience scenario: The presence of the collaborator is represented by means of an avatar representation (Figure 1b). The position of the head and torso of the avatar indicate where the collaborator is looking at, and the avatar's hands reproduce her gestures for non-verbal communication. • Cues for the shared workspace scenario: Taking into account the results of the pilot test we do not include an avatar representation of the collaborator, and rely on the awareness cues of actions and location to provide an indirect hint that someone else is working at the same time. (Figure 2b).

2) Awareness cues for actions
We provide auditory feedback for adding and deleting marks. Also, we provide indirect communication cues for the communication. We limited the communication acts during the task to: asking for help to the other collaborator, and showing approval to the other collaborator. We made this limitation considering these two communication acts were the most often used in the pilot test, and the ones that our previous survey indicates that users agreed how to perform the gesture and which emojis best replaced the mentioned communication acts.
It is important to mention that the present study focuses on visual cues to support workspace awareness in collaborative spaces. For that reason, we provide only visual cues to express the mentioned communication acts. The cues provided in each scenario, and their design rationale are: • Cues for the shared experience scenario: The communication acts are represented by the hand gestures of the avatar, which reproduce the movements of the Oculus Touch controllers. Users could choose which gesture to use for asking for help or showing approval in the same way as they would do in the real world. • Cues for the shared workspace scenario: The communication acts are represented by emojis, which users activate by pressing the buttons of the controllers.

3) Awareness cues for location
Both scenarios include a viewport, which displays the area the collaborator is working on currently, and a pointer, which signals the location in the map is pointing to. The cues provided in each scenario, and their design rationale are: VOLUME 4, 2016 -Implicit coordination [10].
-During long time collaboration, users start to ignore avatars [10].
-Difficulty to interpret gestures of the avatar [26].
Pictorial cues for communication (Emojis) -Can represent a variety of concepts like facial expressions, emotion, activities, etc [42].
-Can substitute words in a speech [44].
-The use of emojis (and images in general) are an easy way to communicate actions, moods, etc [4].
-Reduce perception of competence in formal settings [17].
Auditory cues (Voice) -Preferred by the users to solve ambiguities or conflicts [45].
-May can be interpreted as a demand to immediate action [16].
-May produce delay, voice loop, among others issues that may interfere with communication especially during critical responses [47].
-It is considered an intrusive way of communication [16].
-Necessity to warp pointing gestures to improve the other person perception of the gesture in virtual environments [45].

Viewport
-Support navigation of groups in virtual environments [28].
-Occupy less space of field of view of the user [27].
-Divert gaze to a specific part of the scene [27].
• Cues for the shared experience scenario: The view area of the collaborator is shown in a viewport displayed in one of the screens of the workstations (Figure 1b). The pointer is represented by a ray that goes from the avatar's hand to the position of the map the user is pointing to. • Cues for the shared workspace scenario: The view area of the collaborator is displayed in a translucid area at the bottom of the map (Figure 2b). The pointer is represented by a cursor that moves over the map. Table 2 summarizes the different representations used for the elements of the map interface in the two scenarios. It is important to note that in the two cases the users have all the visual cues at their sight, so the amount of effort to check them was similar.

1) Shared experience scenario
This scenario reproduces the setting of a crisis room scene (Figure 1a). In the front wall of the room a big map displays marks depicting the location of certain incidents during an emergency crisis. In the middle of the room there is a workstation with 3 screens (Figure 1b): the first screen displays information about the incidence the user is pointing at in the map, the second displays a view-port depicting the current area of the map the other collaborator is looking at the moment, and the third one was designed to have it unused and shows a set images depicting pictures of the incidence pointed. The scenario includes an avatar representation of the collaborator, which reproduces her gestures and movements as they are captured by the Oculus Quest HMD and the Oculus Touch controllers she is equipped with.  In this scenario, we only include the map screen (or the workspace), which occupies the entire field of view of the Oculus Quest HMD (Figure 2a). In this case the scenario does not include an avatar representation of the collaborator. A supporting screen (translucid area at the bottom left corner of the map workspace Figure 2b) displays the viewport of the collaborator, the information about the incidence and some emojis that the collaborator can manually activate using the buttons of the Oculus Touch Controllers. In case of the emojis, the user can visualize her own emoji at the top of the supporting screen, and the other's emoji at the collaborator gesture zone (see Figure 2b). Furthermore, the available emojis were two: a thumb-up-emoji and a handpointing-up emoji which represent the two communications acts available for the experiment (asking for help to the other collaborator, and showing approval to the other collaborator).

V. EXPERIMENT DESCRIPTION
In the following section we describe the study in detail: the design of the study, implementation and setup, participants, task, and procedure.

A. STUDY DESIGN
The experiment was designed following the guidelines and recommendations of the Ethics Committee of the university which approved the experiment. The study design was within-subjects that allows us to observe the behavior of the same couple of collaborators using both scenarios. We considered as independent variables two mechanisms for workspace awareness: shared experience scenario, and shared workspace scenario. The dependent variables of the experiment were: task efficiency and task workload. In addition, we also analyze the perception of the user in terms of: co-presence, attention allocation, perceived message understanding, perceived behavioral interdependence, and perceived efficiency (following the social presence questionnaire proposed by Harms et al. [20]).

B. IMPLEMENTATION AND SETUP OF THE MAP INTERFACE TOOL
For the experiment we used an implementation of a collaborative map interface developed using the Unity game engine. The entire ecosystem consisted of a pair of Oculus Quest HMD and Oculus Touch controllers, which the participants used to interact with the VR environment, and a computer that permits the researcher to track the experiment and collect the data. The communication of the entire ecosystem was facilitated using the Photon Engine 4 .
The experiment was carried out in pairs at the same lab, but as each participant was using an HMD they could not perceive the physical presence of her companion. Furthermore, participants were seated approximately 1,5 meters from each other during the experiment as shown on Figure 3.
Finally, each collaborator has its own personal map, which means that each collaborator has its own zoom and position of the map independently the zoom and position of the other collaborator, but both participants were manipulating the same map and they can see the other's action by using the viewport or going to the same geographic position of the other.

C. PARTICIPANTS
For this experiment we recruited twenty four participants (9 females, 15 males). The average age of the participants was 24.83 years (stdev = 5.72). The profile of the participants were: undergrad students (10), master students (5), PhD students (7), postdocs (2). The majority of the participants come from STEM fields and three of them from Economics. Thirteen participants had no previous experience with VR, and only two of them use this technology regularly. All For the execution of the experiment, the participants were matched following their schedule availability, and we did not take into account whether they knew each other.

D. TASK AND PROCEDURE
In the proposed task two participants have to collaborate identifying and marking targets that appear every five seconds on certain locations in the map. In the task, the participants have to navigate the map, zoom in/out, point to the targets, read their info, and add marks on them when appropriate.
To clarify the task we set it up in the context of the scenario of an emergency crisis intervention. Each participant was assigned to one role: security agent or health-care personnel. The targets represent incidents that need to be attended and solved. When a participant finds an incident and points at it, a description of the incident is displayed ( Table 3) providing clues of the role that has to attend it. Some incidents should be attended by security agents, others by the health-care personnel and some others require the assistance of the two roles. In this way three possible cases (of individual and collaborative work) might occur when a participant identifies an incident: • The incident can be solved by the role who found it: In this case, the participant adds a mark at the incident location to sign it as solved. • The incident should be solved by the other collaborator: In this case, the participant should notify the other collaborator that she found an incident for her. • The incident should be solved by the two collaborators: In this case, the participant adds a mark at the incident location and should notify the other collaborator to do the same to sign it as solved.
During the experiment, participants had access to the description of the incidence (located in one of the screens of the work station in the case of the shared experience scenario or in the supporting screen on the shared workspace scenario) if the zoom level was greater than 6 points over 20 points of zoom available for the used map (using this level of zoom participants can visualize the names of cities of a country in the map) otherwise a message saying "Your are to far away of the incidence was displayed". These descriptions appeared every 5 seconds as a pin-mark over the map (the type of incidence, possible cases, and geographic position were selected by the system randomly in order to avoid bias caused by experimental procedures like learning effect or fatigue [29]). The total sample of incidences at the end of each experiment was 60 incidences. At the beginning of each trial, each participant is assigned to one role. A member of the research team explained the task, how to interact in each VR scene, how to display the social cues and, in general, make sure they understand how each element of the application worked. Also, they were given time to practice with the system until they were ready to use it. The entire protocol for running the experiment is detailed in Table 4. The entire experiment took approximately 25 minutes per couple of participants. Furthermore, in order to avoid possible bias caused by experimental conditions like learning effect or fatigue, the order in which each pair or participants worked first with each scenario was counterbalanced.
As mentioned in section IV.A.3, to allow us to investigate the differences between the two non-invasive and visual rep-resentations of the communication acts and isolate them from other factors that could negatively affect the results of the experiment, we encourage the participants to use the visual cues for communication instead of the voice. It is important to mention here that the main purpose of the experiment is to compare visual representations and, hence, we isolate visual cues for other ways of representing it like auditory cues, or haptic cues. Step Description 1 Greetings, experiment introduction, and sign of the informed consent.
2 Tutorial for introducing the user to VR Interaction. 3 Training session for both scenarios. Pre-questionnaire responses for weighting the importance that each participant has for every dimension of the NASA-TLX Questionnaire.

5
Random assignment to one scenario for the first trial. 6 First trial (execution of the task for a fixed time of 5 minutes).

7
Data gathering from the interface. 8 Post-questionnaire responses (NASA-TLX and Social presence questionnaires).

9
Second trial (execution of the task with the other scenario for a fixed time of 5 minutes).

10
Data gathering from the interface.
In the next section, we detail the data collected during the experiment, and the instruments used to gather it.

VI. DATA COLLECTION
In this experiment we collected quantitative and qualitative data from the participants to connect the results of the objective data (efficiency and task workload) with the perception of the users about the two scenarios.

A. QUANTITATIVE ANALYSIS 1) Performance
For performing the quantitative analysis, we calculate the task efficiency of each couple of participants considering the numbers of marks added to the map. We distinguished, and analyzed separately 3 types of marks: successful, incomplete, and wrong marks. This approach was done to identify the amount of collaboration in each scenario representation as incomplete marks will show a lack of collaboration between the users.
Successful marks are those added in a radio no more than 10 units (latitude and longitude expressed in degrees) of the geographic coordinates of an incident by the corresponding role that had to attend it. Marks found in a zone outside of this radio were considered wrong marks as they are too far from the incidence to be considered as successful marks. It is important to notice that this radius distance threshold was the same for both scenarios representations regarding the size of the map as the geographic units are independent from the size of the map.
For those incidents that had to be attended by the two roles, it was necessary to find the two marks in place, otherwise the mark was considered as incomplete.
Finally, wrong marks are those added to the map but with no relation with an incidence.

2) Task workload
For measuring the task workload, we used the NASA-TLX [21] questionnaire. This instrument provides a weight of the effort required to perform a task based on the 6 dimensions: mental effort, physical effort, time required, degree of failure, effort, level of stress.

3) Social presence and Perceived efficiency
To study the potential effects of social presence in the participants' efficiency and workload we used a selection of questions from the social presence questionnaire [20]. Additionally, we added a question for measuring the perceived efficiency of the participant. The items of this questionnaire are detailed on Table 5. Participants ranked each item in a likert scale from 1 (Totally disagree) to 5 (Totally agree).

Perceived efficiency
In the task that we carried out together with the other person, I felt that the collaboration was adequate and we fulfilled the task that was asked of us.

B. QUALITATIVE ANALYSIS
At the end of the experiment, we interviewed the participants to collect their impressions about the two scenarios. We recorded each interview and codified their responses using thematic analysis [5]. We state the following questions to the participants: 1) Give your general impressions of the two scenarios 2) In which scenario did you feel a greater presence of your partner and why? VOLUME 4, 2016 3) In which scenario did you feel a greater task workload and why? The entire analysis and codification of the data extracted from the interviews was performed individually and then triangulated by two of the authors of this study.

VII. RESULTS AND ANALYSIS
For analyzing all dependent variables, a Wilcoxon signedrank test was used with a significance level of α = 0.05. A Shapiro-Wilk test indicates the non-normal distribution of the data (p < 0.001).

A. TASK EFFICIENCY
For analyzing the task efficiency of each scenario, we detail and present the results of the number of successful, incomplete, and wrong marks. Figure 4 summarizes the mean of successful, incomplete, and wrong marks. The Wilcoxon Signed-Rank Test indicated that the successful marks for the shared workspace scenario (M dn = 9.50) was statistically significantly higher than the successful marks for the shared experience scenario (M dn = 6) Z = 169, p < 0.017.
Furthermore, the Wilcoxon Signed-Rank Test indicated no statistical significant difference for incomplete marks (T = 60.5p < 0.97) and wrong marks (T = 84.5p < 0.38). Figure 5 summarizes the raw scores of the NASA-TLX questionnaire. The Wilcoxon Signed-Rank Test indicated that the NASA-TLX weighted workload for the shared experience scenario (M dn = 40.33) was statistically significantly higher than the Nasa-TLX weighted workload for the shared workspace scenario (M dn = 36.67) T = 58.50, p < 0.009. Figure 6 summarizes the participants' responses to the answers of the social presence questionnaire. The results of the Wilcoxon Signed-Rank Test to each answer are the following: Test indicated that the median for this question of the shared workspace scenario (M dn = 5) was statistically significantly higher than the the median for this question of the shared experience scenario (M dn = 3.50) T = 149, p < 0.026.

D. THEMATIC ANALYSIS FROM PARTICIPANTS INTERVIEWS
The purpose of this qualitative analysis was to collect the impressions and feelings of the participants, and to establish a connection between their responses in the interview with the quantitative data collected in the previous section. For this reason, we codified the interviews and clustered their responses into the most representative categories of the quantitative analysis. The patterns of comments given by the participants are detailed in Table 6, the number at the beginning of each comment represents the number of participants that gave similar comments. In the following sections, we present the results and examples of the comments given by the participants.

1) Social presence and communication
A frequent comment was that even though the presence of the avatar is a strong cue of the presence of another person, the feeling of being with another person is also felt in the other In the case of the shared experience scenario, the majority of participants indicated that the avatar gives a strong feeling of co-presence ([C1P1]: "Seeing the avatar makes me feel that there was another person with me"), but other comments also mentioned that because the variety of gestures that they can perform, the communication was sometimes cumbersome ([C6P2]: "It was more complicated to understand the gestures of the other", [C3P2]: "When I looked back the other, I felt that he always had the same gesture").
In the case of the shared workspace scenario, many participants mentioned that it was simpler to work as [C10P2] said "It was easier using this scenario because of the emojis, they facilitate the communication with the other"). It was also mentioned that emojis allowed the participant to give faster answers to the other and also to catch the message of the other faster ([C8P8]: "I think that I like this scenario because of the emoji. With it, I already knew what the other person was telling me"). In terms of co-presence of the other collaborator using a shared workspace scenario, some participants mentioned that they become aware of the presence of the other person only when the other person requested for help ([C5P1]: "In this scenario, you know the presence of the other when she calls your attention"). Other participants mentioned that they felt more co-presence just because the communication was easier ([C6P1]: " I noticed 5 Identifier of the subject C represents the couple and P the participant the presence of the other person just because it was easier for me to see the emojis of the other person").

2) Representation of the scenario
In the case of the shared experience scenario, participants indicated that the elements of the scene make the task more interesting and entertaining ([C6P1]: "In this scenario the scene is beautiful because you can see more things, I like this"), but they also indicated that this elements make them less efficient ([C5P1]: "The room has a lot of things that could distract you like the the desks, phones, or the avatar"). Participants also indicated that they had higher workload as they had to allocate their attention in the elements of the scene ([C7P1]: "This scenario required to be more aware of the space and the other person").
In the case of the shared workspace scenario, participants found useful the convergence of some of the awareness cues in the left-down corner of the map because it was more efficient and they did not have to divert their gaze as much as the other scenario. A great number of participant (14) agreed that working in this scenario was more efficient ([C11P1]: "I had the tools available at my sight, it was more focused and intelligible", [C7P2]: "I had fewer things to attend. I had only to wait for the call of the other through the screen and respond to them with an emoji").

3) Workload
One issue identified by the participants in both scenarios is that they had a secondary task to perform besides attending the incidences. This secondary task, intentionally included in the design of the experiment to test our hypothesis, consists of being aware of the intentions and requirements of the other collaborators. ([C3P1]: "I had to constantly divert my attention from the main task to the intentions and requests of the other").
In the case of the shared experience scenario, participants found that they have to put a slightly more effort than the other scenario because of the variety and range of the body movements when they are communicating with the other collaborator ([C4P1]: "Talking about the gestures that I have to perform for communicating to the other, I think I had to put a little more effort because I had to move my arms to perform the gesture").
In the case of the shared workspace scenario, many participants indicated that the supporting screen was helpful for executing the task ([C2P2]:"I found it useful having all the tools in the same screen").

VIII. DISCUSSION
The results of the experiment reject our initial hypothesis. From the statistical analysis, we can suggest that: H1: Participants attend more incidences and collaborate more efficiently in a shared workspace scenario. H2: Users perceive lower levels of workload in a shared workspace scenario.  (9) The variety of gestures for communication make it cumbersome.
(5) The elements of the scene make it entertaining and interesting.
(15) The elements of the scene make them less efficient.
(9) The variety and range of the body movements make the communication harder to perform.
Shared workspace (9) It was easier to communicate using emojis.
(4) I become aware of the presence of the other person only when the other person requested for help.
(5) I felt the presence of another person just because the communication was easier.
(14) The distribution of elements of the scene make them more efficient.
(11) The supporting screen was helpful for the task.
Both scenarios (8) I felt the presence of another person in both scenarios.
No representative comment to report. These results suggest that the shared workspace scenario provides better support to collaboration in terms of efficiency and workload when compared with a shared experience. These results are also in accordance with the participants' commentaries during the interviews. Furthermore, the results also suggest that participants did not perceive greater difference in terms of co-presence between the two scenarios. In the following subsections, we summarize the main findings of the experiment.

A. EFFICIENCY AND WORKLOAD
The results of the study suggest that, in terms of efficiency and workload, the shared workspace scenario performs better. This can be explained because of two main reasons. On the one hand, even though the participants appreciated the realism provided by the shared experience environment, the collaborators had fewer peripheral elements in the shared workspace scenario that could interfere and distract them from the main task. On the other hand, the communication using emojis was simpler and more straightforward as there is no need to interpret the gestures of the collaborator's avatar.
The quantitative results indicate a greater number of attended incidences in the shared workspace scenario. These results are in accordance with the comments provided by the participants who indicated that they felt more efficient in the shared workspace scenario and the communication with the other collaborator was easier and more straightforward.
Furthermore, even though both scenarios were designed in a way that the amount of effort to check the cues were similar, participants appreciated that they were grouped together in a corner of the map in the shared workspace scenario. In any case, it has to be considered that each scenario tries to reproduce a different collaboration paradigm. On the one hand the shared experience tries to reproduce the way that humans collaborate in the real world, where the user could have visual cues in their peripheral view. On the other hand the shared workspace tries to reproduce collaboration using abstractions of the 2D collaborative interfaces, where the user has all the awareness cues in the field of view of the screen.

B. SOCIAL PRESENCE
In general terms, the participants did not report major differences in terms of perceived presence of the collaborator between the two scenarios. The strongest cue to inform the presence of another person in a virtual environment, as in reality, is an almost realistic representation of the body of the human.
Despite the valuable contributions made on the advance of natural interaction methods in virtual environments, natural representations of social awareness cues can be cumbersome and hinder the communication between the collaborators. As technology keeps maturing, traditional social cues used in 2D environments can be used in immersive virtual environments. The results of this study suggest that traditional awareness cues, largely used in traditional 2D collaborative environments, can be enough to give the user the feeling that she is not working alone in the task.
Moreover, according to participants, the gestures and movements of the collaborator's avatar could also constitute an element of distraction from the main task and negatively affect the task efficiency. The results of the study suggest that using graphical representations of the communication acts, as emojis, might be a more suitable approach for indicating actions and intentions to the collaborator. However, it is also necessary to consider that in the study only two communication acts were represented, which were easily activated using the controllers' buttons. In other settings which require a higher number of representations and in which a greater number of collaborators participate, the activation and identification of the communication acts' representations might be more complicated. It would be necessary to investigate how to activate and represent these acts without increasing the workload of the main task.

C. STUDY LIMITATIONS
We are aware of some limitations of our study. In this section, we mention some of the limitations of the study.
First, most of the participants have a STEM background. People with different backgrounds could find it more difficult to use the system. Second, the current implementation of the system only supports the interaction between two collaborators. It would be necessary to further investigate the effects of the virtual representation of the collaborative space as the number of collaborators increases.
Third, the type of task and duration selected for this study may constraint our results, and different ones could produce different results. The type of task and duration selected for the study was chosen in order to isolate our analysis from fatigue, learning effect, or others. It would be necessary to further investigate the effects that complex and longer tasks may produce in collaboration.
Finally, another potential limitation was the recommendation of avoiding the use of the voice for the communication acts of asking for help and showing approval. The intention of this recommendation was to isolate visual and non-invasive ways of communication from the voice that could be intrusive and interfere with the other collaborator when she was doing individual work [16]. It would be necessary to further investigate the effects of other channels beside visual cues for transmitting communication acts during synchronous collaboration.

IX. CONCLUSION AND FUTURE WORK
Immersive technologies constitute a suitable alternative to implement computer supported collaborative work. However, to fully explore the potential benefits of these technologies on collaboration it is necessary to understand how to provide awareness of the actions of collaborators and how to present it in the collaborative space. In this study, we presented a comparative study between two visual representation approaches for map interfaces: shared experience and shared workspace. Despite the fact that a shared experience scenario is expected to provide a better personal experience to the user in terms of realism and social presence, our study provides insights that suggest that a shared workspace could be a more effective way to represent the scenario and improve the collaboration. Furthermore, our study suggests that the participants did not find greater differences between both models in terms of social presence.
This paper contributes with some insights about how to design collaborative map interfaces that effectively support collaboration in immersive virtual environments. The results of this study can also be generalized to other domains beyond map interfaces. For example, domains like telematic events, VR gaming, collaborative design, among others can also use the findings of this paper when designing collaborative interfaces. However, it is necessary to take into account the particularities of each domain field when designing visual cues for providing situation awareness in the collaborative scenario.