Using Portable Virtual Reality to Assess Mobility of Blind and Low-Vision Individuals With the Audomni Sensory Supplementation Feedback

Numerous electronic travel aids (ETAs) to increase the mobility of blind or low-vision (BLV) individuals have been proposed. However, the lack of established and well-motivated methods, and of recruiting enough BLV test participants, keeps a successful aid illusory. To combat this, a new aid-agnostic questionnaire focused on mobility, the Desire of Use Questionnaire for Mobility of BLV individuals (DoUQ-MoB) and a new portable, large-scale-exploration virtual reality (VR) system, the Parrot-VR, were employed to evaluate the ETA Audomni. Through VR and Audomni, 19 heterogenous BLV participants traversed large-scale urban environments. Their experiences were probed through the DoUQ-MoB, and their movement analyzed. Numerous results are presented, a highlight being that most participants, 76 %, responded that it was very or extremely likely that they would want to use Audomni along with their current aid. Further, Parrot-VR assists in recruiting a satisfying number of diverse BLV participants; and DoUQ-MoB allows to systematically probe their opinions of an aid, and how it relates to others aids, in a considerable quantity of mobility aid aspects. This work illuminates some shortcomings of Audomni, but also shows a majority of BLV participants actually wanting to use a proposed ETA — a result rarely seen so distinctly in the field, and which encourages the continuing efforts of the project. The results are supported by a novel test procedure, which might serve as future inspiration to the field.

While ever-increasing ETA proposals and technological development should steadily increase the chances of such an ETA emerging, they are diminished by the fundamental issues pervading the field.Despite attempts dating back to at least the 1940's [18], there are still no established methods or measures used for ETA evaluation [3], [8], [13], [15], [16], [17], and as we have argued elsewhere [19], [20], [21], most used are poorly motivated, offer little insight, and have low reproducibility.Suggested design guidelines and best practices of developing ETAs are also almost consistently ignored, which lead us to develop the Desire of Use model (DoU), and its application on mobility of BLV individuals (DoU-MoB) [20].Further, it is quite rare to assess the user experience or aid acceptance of BLV individuals themselves [8], [10], [13], [15], [16], [17].This is likely due to the last and critical point: that actual blind and low-vision participants are much too often entirely neglected in tests, and when included, it is most frequently in inadequate numbers [8], [10], [11], [15], [17], [22], [23], [24].

A. DESIRE OF USE QUESTIONNAIRE FOR MOBILITY OF BLV INDIVIDUALS AND THE PORTABLE VIRTUAL REALITY SYSTEM, PARROT-VR
To answer these issues, in our parallell paper [19], we propose two tools.First, a patient-reported outcome measure (PROM) 1based on the mobility and aid aspects identified by DoU-MoB, called the Desire of Use Questionnaire for Mobility of BLV individuals (DoUQ-MoB).It features a comprehensive set of 106 items regarding mobility, and as such provides insight about an aid at various levels of abstraction.It is aid-agnostic, i.e. it can be used for any aid, and promotes and facilitates the comparison of aids, e.g. a proposed ETA with a currently used aid.Most of the items are answered once for each aid to be assessed, but there is also a part with items that compare the aids directly, and acts like a semi-structured interview.The DoUQ-MoB leverages the best known practices of survey design to ensure as reliable results as possible, and it has been through one validation round with eight orientation and mobility experts and six BLV respondents.
The second tool is the Portable, Absolute Rotation, Relative (O) Translation VR system, Parrot-VR [19].VR in general allows for excellent reproducibility in that the testing environment, along with external factors, e.g.traffic, weather, and time-of-day, can be entirely controllable and consistent between participants.The control scheme of Parrot-VR, using a head-mounted inertial measurement unit (IMU) and a video-game controller, enables reasonably intuitive large-scale exploration, which makes traversal of relevant urban virtual environments (VEs) possible.Lastly, the portability makes it possible to bring and setup the system almost anywhere, enabling testing in e.g.participants' homes.This considerably expands the group of feasible BLV participants-and eases the recruitment of an adequate amount substantially.

B. THE AUDOMNI ELECTRONIC TRAVEL AID
Audomni is an ETA in the form of a sensory supplementation (or substitution) device that can be used either as a primary mobility aid, i.e. by itself, or as a secondary, i.e. along with a long cane or guide dog [21].The aim of an SSD is to translate one sensory modality to another or several others.For SSDtype ETAs, this is predominantly visuo-auditory, visuo-tactile, or hybrid supplementation; meaning that visual information is in some way translated to audio, touch, or both.Whereas most ETAs tend to limit their feedback to for instance obstacle detection or path finding, the aim of SSD-type ETAs is often to provide a much greater sense of users' environments.
Audomni is a visuo-auditory SSD that utilizes a head-mounted depth camera; translates the data into sound through various algorithms, e.g.coding horizontal angles to stereo pan, vertical angles to pitch, and distance to volume; and provides the resulting audio through bone-conductive headphones.These feedback algorithms have been modified to better allow for horizontal and vertical differentiation since our previous publication [21], though the fundamental principles remain.

C. OVERARCHING PROJECT AIM
DoU states that the desire to use an available aid (or combination of aids) for a specific activity, is a function of its functionality and ergonomic complexity. 2,3If the functionality is increased, or the ergonomic complexity decreased, DoU increases.If such a DoU of a new aid (combination) exceeds the one of a current aid (combination), the user will opt for the new one instead.Thus, in order to produce an aid that results in an improved mobility for BLV individuals, DoU yields the following assumptions:

Project assumptions A1: Increased mobility increases BLV individuals' quality
of life and decreases their associated socioeconomic costs.A2: An aid, alone or as supplement, increases users' mobility if used and the functionality result is much greater than that of a current aid (combination).A3: An aid, alone or as supplement, is used if the resulting functionality and ergonomic complexity yields a greater DoU than that of a current aid (combination).
This leads to the hypotheses, as well as design goals, required for increased BLV user mobility:

Project hypotheses and design goals H1:
The aid, alone or as supplement, results in a functionality much greater than that of a current aid (combination).H2: The aid, alone or as supplement, results in a functionality and ergonomic complexity yielding a greater DoU than that of a current aid (combination).
To demonstrate these hypotheses for any ETA adequately, long-term user tests are likely required.This so that the effects of novelty and inexperience are mitigated and potential new habits can be formed, and sufficiently strong metrics can be collected.Long-term tests are inherently dangerous for the participants, and all precautions to ensure a safe and dependable aid must therefore be made.This work can be regarded as one step of many to achieve an ETA proposal ready for long-term tests, and thus does not attempt to show H1 and H2.

D. AIM OF PAPER
While Audomni then is not yet ready for evaluating H1 and H2, it is still imperative to assess any proposed ETA regularly; both to estimate how far away such evaluation is, and to make sure that the ongoing development is correctly focused.Meanwhile, to maximize the potential success of any ETA to increase the mobility of BLV users-not just Audomni-it is of great importance to make efforts in alleviating the evaluation issues of the field outlined in Section I. Thus, the aim of this paper is twofold: 1) take stock of the current Audomni feedback; and 2) evaluate the benefits and limitations of the DoUQ-MoB, the Parrot-VR, the test procedure utilizing them, and the various analyses herein.Preliminary versions of this work has been reported in [25] as well as in [26].

1) A NOTE ABOUT TERMINOLOGY
There is some discrepancy in the field in how various ETAs are referred to.For instance, sometimes ETAs and electronic orientation aids, EOAs, are distinguished; and sometimes ETAs, EOAs, and position locator devices (PLDs).In this work all proposed systems are simply considered ETAs, with a distinction instead being made if it is a primary or secondary mobility aid.A primary mobility aid should be sufficient for navigation by itself, whereas secondary mobility aids should be be used in conjunction with a primary.Examples of secondary mobility aids are handheld, wearable, or canemounted distance sensors with haptic or auditory feedback; GPS systems; or scene description applications.

A. PARTICIPANTS
19 self-identified BLV test participants from 14 different municipalities in eastern and southern Sweden were included in the study.Their distribution is shown in Table 1.They were recruited through various BLV advocacy groups announcing the study to their members, and in some cases through snowballing where such members shared the announcement with personal contacts.
All of the participants performed the VR tests, see Section II-E, and completed a version of the DoUQ-MoB, see Section II-D.However, after experiencing clear issues with the DoUQ-MoB after having administered it to the first two participants it was revised, see [19].This revised version was then used for the remaining 17 participants, and are what is reported for the questionnaire items in this work.Of these 17 respondents, 14 answered as cane users, and 3 as dog users.The semi-structured interview was analyzed for all 19 participants.
All participants gave their informed consent for inclusion before they participated in the study; and the study was conducted in accordance with the Declaration of Helsinki and it, including the tests, participant recruitment and consent procedure, as well as data management, were reviewed and approved by the Swedish Ethical Review Authority (diary number 2021-04835).

B. THE VIRTUAL ENVIRONMENTS
To maximize learning outcomes the participants should be subjected to highly relevant scenarios.Therefore, we based the VEs on the important mobility subtasks identified by the DoU-MoB [20].As including all these certainly would yield both intense and unrealistic testing scenarios, and as the Parrot-VR currently is arguably more suitable for micro-scale navigation, we focused on the DoU-MoB subtasks: detecting obstacles (+ to the sides), traveling surface level changes, true paths, objects at traveling surface level; identifying landmarks; and apprehending nature of objects (+ to the sides).
Such VEs should facilitate insights of how Audomni aids in these subtasks through the analysis of users' movement through them, and of any collisions or other events that occur.Crucially, it also provides the BLV participants a first-hand account of this, which then can be systematically collected using the DoUQ-MoB.They can also extrapolate how Audomni might actually aid in the excluded DoU-MoB subtasks as well, e.g.handle public transport, 4 for which they could provide less certain, but nevertheless useful feedback.
We argue that a train station area is a highly relevant setting, a notion underpinned by the handling public transport subtask, and since such often feature ample instances of the other subtasks to be included.It would also provide a natural scenario for the participants, where tests can simulate arriving at the station area by train, and leaving it on foot.
Basing the VEs on a real-life train station reinforces both the relevancy of them, and possibly the participants' ability to assess how the ETA might work for them in reality.It is also convenient from a VE production perspective [19].
We built the VE by replicating the central station of Lund, Sweden, as it features the desired mobility subtasks; it was in our vicinity, which aided in the modeling process; and it offers the opportunity to perform VR-to-reality tests in the future.
To keep an individual VR session from becoming too lengthy, as well as to aid in the training part of the test, the VE was split into three scenes.They depict the user: 1) traversing the train platform by coastlining (aka shorelining) the platform edge; 2) after leaving the train platform, move as straight as possible to a street curb without a consistent coastlining opportunity (beeline); and 3) coastline the street curb until they reach an unsupervised pedestrian crossing.
Two versions were made of each scene.One was for introducing and training with the ETA, and featured very few obstacles or difficult passages, as well as some parts tailored to teach navigation strategies.The other version was for testing, featuring numerous obstacles of different sizes and at various heights, as well as passages that were difficult, narrow, or both.Indeed, for some obstacles and passages it was all but known that participants should not be able to navigate them with the Audomni feedback without either intervention, or by avoiding them by chance.These were still included in an effort to shield us from making biased tests toward Audomni, as well as from the known issue of ceiling effects from all-too-easy tests in the field.The resulting VEs are shown in Fig. 1-3.

C. MOVEMENT ANALYSIS
The recording affordances of VR provide an opportunity to perform a movement analysis, which should help guide ongoing design efforts.As it is difficult to provide quantitative measures which can elucidate potential issues or advantages if there is no baseline measure, we opted for qualitative studies.
First, a systematic event analysis was performed for all test scenes.All collisions or other events where the test administrator intervened were identified in the merged recordings of the VR movement, Audomni feedback, and speech, see [19].Then, for each collision the following parameters were noted:

Registered collision event parameters
For collision events, the feedback behavior [warned, did not warn, warned initially but not at collision time], underlying reason(s), collision angle [collided with the object head-on, grazed the object, collided with the object backwards], speed [low speed, high speed], test number, time, participant, and specific obstacle were registered.The reasons were based on our judgement of the VR movement, Audomni feedback, as well as any speech before, during, and after collisions.The angle was deemed a head-on collision if it was < 45 • , and the speed was deemed high if it was consistent with the participant's overall walking speed.Afterwards, each reason is coded as if it was due to a feedback or method issue; if it might be both, it is coded as a feedback issue.A feedback issue would be one where the nature of the feedback (or lack of feedback) could explain the reason for the collision; a method issue one where e.g.issues with the VR system or how the tests were performed should explain it.Each obstacle was also coded into a ground-level change, low-level obstacle, head-height obstacle, or other.Lastly, a collision was classified as a method collision if all reasons were method issues (e.g. the participant collided while being guided by the administrator, or the sensor FoV was tilted due to poor calibration); a feedback collision if any was a feedback issue (i.e. the feedback was working as intended, but a collision happened anyway); or a severe collision if it is a feedback collision occurring at high speed, or with a ground-level change (e.g. the train platform edge and the broken gutter), low-level obstacle (e.g. the low wall and scooters), or head-height obstacle (e.g. the construction and loading platforms).
Other interventions were generally due to the participant needing guiding in some manner.A similar event registration as for collisions was done for each of these ''guiding'' events: (Bottom) the first virtual test scene which has added obstacles, hazards, and narrow passages.The object with a blue field protruding from it is the participant's virtual avatar, and it denotes the starting positioning of the scene as well as the field-of-view of the virtual electronic travel aid (ETA).The red dotted lines denote severe obstacles.The goal in these scenes is to reach ''Steps down 2'' by coastlining ''Platform edge''.Note the increase of obstacles and narrow passages in the testing scene, as well as that ''Gutter 3'' has been replaced with ''Broken gutter'', which extends to the middle of the platform.Whereas the regular gutters have a grating with 2 × 10 cm bars spaced 2 cm apart, the broken section has the same grating but with two bars missing at regular intervals of 44 cm.This represents holes of 10 × 10 cm, arguably enough to be a tripping hazard, and should thus be avoided.(The scenes have been slightly edited for visual clarity of this figure.)Figure and caption originally produced for this work, but published first with slight differences in [26].

VOLUME 12, 2024
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

Registered guiding event parameters
For guiding events the prompt, underlying reason, provided guiding, test number, time, participant, and specific area were registered.A prompt is what made the test administrator intervene, and prompts, reasons, and provided guidings were judged as the collisions event reasons, see above.An event is then classified as a method guiding if all reasons were method issues, or as a feedback guiding if any reason was a feedback issue.
The event analysis can be visualized to further identify issues with the Audomni feedback or the method.River (aka Sankey) diagrams were produced using the Plotly graphing library for Python [27], [28].One diagram depicts all collisions, broken down according to type of obstacle, specific obstacle, and collision speed, after which the underlying reasons are grouped according to if it was a feedback or method reason.The other diagram illustrates all guiding events, broken down according to specific area, and then into underlying reasons like for the collisions.
Heat or trace maps for movement analysis have seen use in other works, e.g.[29], [30], [31], [32], and [33], and can provide useful results.Using the numeric computing platform Matlab [34], the VR recordings, and the event analysis, participants' movement is traced on a VE in planar view, the traces colored blue for independent walking; yellow for guided walking due to a method issue; or red if due to a feedback issue.Next, all collisions are overlaid with yellow circles denoting if due to method issues; red circles if due to feedback issues; and red crosses if they were severe.To increase clarity, the resulting heat map are split into two, one with the blue tracesindicating what users are capable of with Audomni feedback; and one with the yellow and red traces-showing potential points of improvement to the method and feedback.

D. DESIRE OF USE QUESTIONNAIRE FOR MOBILITY OF BLV INDIVIDUALS
To systematically and thoroughly probe participants about how Audomni, and their current primary mobility aid, make them handle important mobility aspects and how they perceive the aids, we employ the DoUQ-MoB survey [19].Example questions and the general format of the DoUQ-MoB can be seen in Fig. 4.
The results are aggregated in paired histograms for items answered for each aid, and single histograms for items directly comparing the aids.The paired histograms are tested if they differ significantly (α = 0.05) with a Wilcoxon signed-rank test, which is also controlled through the Holm-Bonferroni procedure.As there is a sizable quantity of the paired histograms, the ten most notable are presented, determined by the lowest p-values.
Hereafter, when a DoUQ-MoB item is referred to, it will be of the form: [ Question prompt, answer], where answer is a term which can be prefixed by one of five strength qualifiers, the as far as we know closest Swedish equivalents to: not at all, slightly/a little, somewhat/moderately, very, and extremely [19], [35].

E. TEST AND TRAINING PROCEDURE
Most tests were performed at participants' homes, except for: one at our department; one in a community locale near the participant's home; and three at participants' workplaces.A participant was first verbally administered an introductory questionnaire regarding the information shown in Table 1.Next, the Parrot-VR system [19], and auxiliary hardware and software were setup, typically as shown in Fig. 5.The participant was fitted with the IMU headband, and both they and the test administrator put on their headphones.
In the first training scene, see Fig. 1, the Audomni feedback was introduced.The administrator described the setting and used the controller to place the VR avatar in front of the train platform edge.While narrating distances and speed, the avatar was moved toward the edge until very close, where it was also turned to the left and right to display such corresponding feedback as well.This was also repeated in front of a building wall.
The avatar was then again placed in front of the train platform edge, and the participant given the controller.They were instructed how to walk forward and backward with the directional buttons, and then to walk until the platform edge was perceived through the Audomni feedback.Next they walked until very close, as previously demonstrated, and then continued until the controller vibrated harshly-indicating that they gone too far.This procedure was also repeated once in front of a bench.
Next, the administrator took the controller to enable headtracking.The avatar was backed away from the bench, though keeping it perceivable.Then the IMU was calibrated [19].(This was done at the start of the latter scenes.)The participant was then instructed to look around, with their head only, and listen to how the Audomni feedback of the bench changed accordingly.
After this, the participant was to take full control.First, the avatar was turned toward the platform edge and the participant given the controller again.Then they were instructed to approach the edge until perceivable through the Audomni feedback; next to turn their entire body to understand themselves parallell to the edge; and lastly, to walk along (coastline) the platform edge to the end of the platform.Considering the available controls for the participant through the IMU and controller, they were then able to move freely in the VE, and it was implicit that they were to react to and avoid any obstacles they might happen upon.During walking, they were advised to think aloud if the feedback changed, as well as to veer toward the edge if they perceived moving away from it, and vice versa.Additionally, they were encouraged to ask any questions, and various situations were described to them as they passed by.This was also the case in the second and third training scenes.In the second scene the participant was to walk until a street curb might be perceived through the  Isaksson-Daun (manuscript to be submitted [19]), but published first in [26].Isaksson-Daun (manuscript to be submitted [19]), but published first with slight differences in [26].
feedback.In the third scene, they were to coastline that street curb, until they reached a pedestrian crossing, recognizable as it is located immediately after a low wall (denoted ''Street wall'').However, in that scene, obstacles are placed to force the participant away from the curb; this to teach the desired strategy of keeping an obstacle within perceivable range in order to walk around it.
While following this described introduction and training procedure, the administrator was to try and help with any misunderstandings and difficulties any particular participant might have had.The introduction and training took approximately 40 minutes to complete.
After the third training scene, the testing proper began, for which speech, movement, and Audomni feedback were recorded.The participant was subjected to the more difficult versions of each scene, see Fig. 1-3.Further, the administrator remained silent unless a collision or fall occurred, the participant seemed to require guiding, or if help was requested.If so, the least intrusive intervention possible was made, from minimal verbal guidings, e.g.''the goal is to find the street curb''; to the very intrusive borrowing of the controller to move past the current scenario; or in three cases spread over two participants, prematurely ending the scene-when there was risk of not having enough time for FIGURE 6. a) Scene 1 movement trace of a theoretical participant walking in an ideal manner according to how the walking instructions for the participants were formulated.b) Scene 1 movement traces for all participants where they managed to walk on their own volition using the Audomni feedback.c) Scene 1 movement traces for all participants where they had to be guided by a test administrator.''Broken gutter'', as well as at the passages ''Construction site 1''-''Steps down 1'' and ''Platform edge''-''Construction site 2'' were clearly troublesome, yet note that a few participants managed even the latter.Figure and caption originally produced for this work, but published first with slight differences.in [26].
the DoUQ-MoB.The testing part lasted for approximately 30 minutes.
The DoUQ-MoB was administered after the testing scenes.For each item in the first part the participant answered once for Audomni, and once for their chosen mobility aid.They got to answer for any aid that they had experience with, not necessarily their current main one.In the second part, which was also treated as a semi-structured interview, they got to answer as how Audomni compared to their chosen aid, and for each item they were asked to elaborate on their answers, and the administrator asked any pertinent follow-up questions.In questions regarding the physical properties of the aid, for Audomni they were asked to answer as if the device had the form of the IMU headband with its cable, along with an imagined waist-worn unit the size of a pair of stacked smartphones.The questionnaire took slightly over an hour to administer on average.

III. RESULTS
Movement traces are presented in Figs.6-8; the event analysis breakdowns, condensed for clarity, in Figs. 9 and 10; the ten cane-Audomni-paired questionnaire items with the lowest p-values in Fig. 11; and comparison items in Fig. 12.The interview coding framework, identified categories, and example quotes are shown in Table 2.The paired questionnaire results for guide dogs can be found in the supplementary material as there were only three such respondents.Beside these, the supplementary materials include video files of the VR recordings with event analyses; the full event analysis breakdowns; individual participant traces; the responses of all questionnaire items; as well as interview transcripts, encodings, and categorizations.The mean durations of the training and testing sessions was 39 min and 30 min, respectively.

IV. DISCUSSION
This section is structured as follows.In Section IV-A, results regarding Audomni from the movement analysis, as well as from the DoUQ-MoB, including interviews, are discussed; then, in Section IV-B, the methods themselves are considered: the VR-system and test participants, the VR-environment, the DoUQ-MoB, and the test procedure.where they managed to walk on their own volition using the Audomni feedback.c) Scene 2 movement traces for all participants where they had to be guided by a test administrator.There were some issues in the ''Construction platform'' passage, and a considerable amount at the ''Bicycle group''-''Stroller group'' passage.

A. DISCUSSION-AUDOMNI 1) MOVEMENT ANALYSIS
The collision breakdown illustrates a number of reasons of severe collisions, see Fig. 9.The most prominent, that the ''warning disappeared. . .'', could sometimes have been resolved with a greater sensor field-of-view; however, most of these collisions were with ground-level changes and low-level obstacles, where this might happen due to the other collision reason ''the user's body obscured the object from the sensor volume''.Thus, this might be an externalization issue, as the location of an obstacle ideally should be remembered even after a warning has subsided.Yet, it is notable that all such collisions occurred at low speed, which could imply awareness of the obstacle but not its exact location; indeed, 82 % of all collisions occurred at low speed so this might be an encompassing issue.However, many occasionally walked slowly when there were no immediate obstacles, which also contributes to this number.
Other striking feedback reasons due to their frequency were ''being exposed to large amounts of sound'' and the related ''getting feedback from multiple objects'', which caused many severe, and most, collisions.Hearing fatigue or cognitive load are potential causes; a notion supported by that [ The amount of information the aid conveys is, tiring] and [ Using the aid for an extended period of time in known environments is mentally, tiring] from the DoUQ-MoB with cane users, both significantly (pre-control) favored the cane, see Fig. 11.Yet, a possible hearing fatigue might not be substantial.For instance, the latter item in unknown areas actually indicated a slight edge for Audomni, with a somewhat lower top bin ratio (see the supplemental material); and for these items, Audomni had no medians of FIGURE 8. a) Scene 3 movement trace of a theoretical participant walking in an ideal manner according to how the walking instructions for the participants were formulated.b) Scene 3 movement traces for all participants where they managed to walk on their own volition using the Audomni feedback.c) Scene 3 movement traces for all participants where they had to be guided by a test administrator.Most issues were at the ''Truck''-''Station house'' passage, but there were also a few incidents at ''Scooters''.
responses worse than moderately tiring.Regarding multiple objects, reasons might be externalization or differentiation difficulty.
The trace maps makes visible a number of troublesome obstacles and areas, see Figs. 6-8.The most frequent severe ''collision'' was with ''Broken gutter'' (see also Fig. 9).In fact, all participants that encountered it failed to avoid it.As the gutter is a ground-level change, earlier points regarding that the ''warning disappeared'' are relevant here as well.However, the gutter is also particular as participants might have learned to ignore it.This since ''Gutter 1'' and 2 (and 3 in the training scene), all produce Audomni feedback indistinguishablefor the feedback designer-from that of ''Broken gutter''.As such, these collisions are both due to that the ''warning disappeared'', and that the feedback was identical to one that should be ignored.Hence, a clear issue lies in that the unbroken gutters generate such feedback, and this is foremost what must be addressed.
All areas with over ten guidings, except for ''Broken gutter'', were various passages.(see also Fig. 10).Whereas the path through ''Construction site 2'' was designed to be very difficult (arguably impossible with the current Audomni feedback and instructions), the others were easier.The most frequent guiding reason was ''not being able to identify a clear path between objects''.Participants ''avoiding more types of feedback than warning'', despite instructions not to, had the greatest issues with this; they strived to move away from all potential obstacles, a strategy both reducing collision risk, but also aggravating problems with the differentiation task of identifying passages between obstacles.For these guidings for other participants, such a differentiation task was likely difficult to begin with.The aforementioned possible externalization and differentiation issues could be mitigated with personalized head-related transfer functions, Audomni feedback modification, or both.It is also possible that the VR setting exacerbated it.Though considering the limited training, it is likely that inexperience of the feedback and Parrot-VR is a sizable contributor to the discussed collision and guiding reasons as well, which also holds for ''turning too few degrees. . .'' in Fig. 9, and ''misidentifying feedback'' in Fig. 10.Thus, less frequent collisions and guidings should be expected with more training.

2) DOUQ-MOB ITEMS
Clearly, cane respondents think that it is less difficult to detect head-height obstacles with Audomni, and more difficult to understand the information Audomni conveys-both of which enjoy significant results post-control, see Fig. 11.Note, however, that while the great majority seem to think that the former is very or extremely difficult with the cane (78 %), and almost as many that it is not at all or slightly difficult with Audomni, the median answer for understanding the information of Audomni is only moderately difficult.
An important consideration of the DoUQ-MoB results is that the Holm-Bonferroni control becomes quite conservative due to the number of tests.Thus, for an indicative study, other items should be reflected on as well (albeit interpreted more carefully).Looking at significant (pre-control) tests, that also express a high top-bin ratio, it seems that cane respondents think that Audomni might make them less afraid of failing to detect head-level obstacles in unknown environments.This might be considered expected given that Audomni is proposed to be head-mounted.Next, responses for [ Using the aid in loud, known environments is most often, difficult] indicate that Audomni would be more difficult to use than the cane in loud, known environments.This is also corroborated by three respondents who had interview statements about that Audomni using sound is an issue, e.g. that listening to both the Audomni feedback and an environment might be exhausting.In ETA research, many proposals make use of tactile feedback with the reasoning that audio feedback may mask important environmental sounds for the user.While the passive masking effect of an ETA's audio display itself can be mitigated by opting for over-ear headphones, as with Audomni (as opposed to traditional in-, over-, or around-ear headphones), the masking effect from the sound itself is of more concern.This seems to be reflected in these DouQ-MoB answers.An important consideration is however that in these tests, no participant is actually subjected to loud environments (see Section IV-B1) and the answers could only be supposed.Thus in future tests it is crucial that participants actually experience ETA feedback in loud environments in order to evaluate this aspect properly.Moving along to the next DoUQ-MoB item of interest, it seems like both the cane and Audomni are deficient when it comes to aspects of public transport.While it remains to be seen if there is any aspect of this that the current Audomni feedback does help with (that the respondents did not consider), it is likely that auxiliary modes could be introduced to aid in for instance identifying the correct bus, an empty seat, etc.
The most notable result of any item-and likely this entire work-is that for [ Wanting to use both the tested aid and my current aid simultaneously is,  likely] the great majority (79 % of cane users, and 76 % of all) answered very or extremely likely, see Fig. 12.We consider this a very positive result, and interpret it quite literally as that they want to use Audomni as an ETA, a conclusion which infrequently can be drawn this clearly in the field.This result is reinforced by that the DoUQ-MoB was designed from best-known survey practices and e.g.uses item-specific scales, instead of Likert-type agree-disagree scales which often suffer acquiescence bias [36]; that only the top two bins very and extremely likely are counted; and the high number, and heterogeneity, of participants.Overall, this speaks to the potential of an Audomni-based aid.This is further bolstered by that for two respondents answering a little and not at all likely, respectively, one answered Much rather use the tested aid for [ If I had to choose. . ., rather use my current-the tested aid] for both known and unknown environments, and the other noted that ''I would not have [dog and Audomni] FIGURE 11. Results of the ten most notable paired items for cane users from the DoUQ-MoB, based on lowest p-value.Asterisks in the left and middle column denote a top bins ratio greater than 50 %.In the right column, a single asterisk denote significance (α = 0.05), and two asterisks denote significance after a Holm-Bonferroni control for the family-wise error rate Figure and caption originally produced for this work, but published first in [26].

FIGURE 12.
Comparison item results from the DoUQ-MoB.Asterisks denote a top bins ratio greater than 50 %, and when applicable, left bins (red) and right bins (blue) are for current aid or Audomni advantage, respectively.The cane holds advantage for all items except for ''. . .walk in a safe manner in unknown environments''.Notable, however, is that the great majority of all respondents deem it very or extremely likely that they would want to use Audomni simultaneously with their current aid Figure and caption originally produced for this work, but published first in [26].
[turned] on simultaneously, but I would easily have both with me.'' The comparison part of the DoUQ-MoB shows a slight trend toward the cane for [ The tested aid allows me to walk in a safe manner, worse-better] for known areas, and toward Audomni for unknown.This reflects the interview category ''Situation differences'', see Table 2, and indicates that the respondents think that Audomni is better than the cane for unexpected obstacles.The cane had a slight edge for [ Understanding the tested aid is mentally, less-more difficult], with a great majority saying more difficult, but with a median of only slightly more difficult.Considering the greater perception volume of Audomni, higher difficulty is expected, and that the median was not worse is notable.[ Using the tested aid is physically, worse-better] had a median of in-between, however considering that Parrot-VR affords limited understanding of the form of a physical aid, and how it might affect this item, further analysis is not attempted.
[ If I had to choose. . ., rather use my current-the tested aid] favors the cane, strongly for known areas, and less strongly for unknown.That the cane predominantly would be chosen is likely due to familiarity, symbolic value, and preciseness of its feedback, as per ''Audomni disadvantages'' and ''Method issues'' in Table 2, and as further discussed in Sections IV-A1 and IV-A3.
In this work, the DoUQ-MoB results have been split between cane and dog respondents as the respective aids are quite dissimilar, and affords mobility to different degrees (guide dogs are often quoted as the best mobility aid).However, due to only three dog respondents, any inferences should be performed with great care, and are not attempted here.

3) DOUQ-MOB INTERVIEWS
As shown in Table 2, the respondents had much to say in the interviews.Possible Audomni disadvantages had the most unique respondent-code pairs, where several potential issues with the Audomni feedback were lifted: it might be more difficult to understand, learn, and localize objects with it.As noted in the ''Method issues'' category, the great differences in familiarity between Audomni and participants' current aids should be kept in mind; however, any steps to mitigate these disadvantages should be made, for instance as per the discussion in Section IV-A1 regarding externalization and differentiation.The lack of symbolic value is divisive.As a subset of BLV individuals reject the cane due to associated stigma, the lack of it should be beneficial for them, though might pose a safety hazard.It is feasible that the physical design of another aid could provide some symbolic value, though most likely less-so than the long cane does (even if many respondents had stories of when their canes seemingly went unrecognized).Nevertheless, this reinforces that the physical design should be individually adaptable, as well as that the use of a cane or dog simultaneously could be critical for many.
Most comments regarding Audomni advantages related to the perception volume, which should make possible for e.g. earlier object detection and better detection at various heights.Still, more than half of respondents said that hands-free use is beneficial.That positive remarks were largely isolated to the perception volume is not too surprising; this as the design of the functional method predominantly originates from that most of the mobility discrepancy between BLV individuals and normally sighted individuals, may be caused by the great differences of their respective perception volumes.That most of the design work of the Audomni feedback consist of making such a volume understandable is not necessarily of interest to users, and the feedback itself thus gathers no positive mention-only negative.This phenomena finely mirrors the use of functionality and ergonomic complexity in DoU [20].In relation to users' currents aids, the functionality increase of an expanded perception volume increases the DoU, which might receive user praise, but it also increases the ergonomic complexity, attracting complaints.

B. DISCUSSION-METHOD 1) PARTICIPANTS AND PARROT-VR
Given the field, 19 blind and low-vision participants is a notable result in its own right, especially as they are, what we consider, well-distributed in regards to gender, age, education, as well as vision impairment and onset.As they were recruited and tested in five different regions in Sweden, the portable nature of the VR-system, see Fig. 5, proved very useful in achieving this.Moreover, that such a diverse participant set managed to reliably control the VR system with no prior experience, and the relative speed in which it could be set up and put away, adds to its usefulness.
Heterogeneous participants notwithstanding, they were likely biased to some degree due to the recruitment process.Since they in most cases both needed to receive the study information from an organization, and be interested enough to both read it and reply, many potential recruits were probably sifted out.The remaining were presumably relatively more interested in new aids, their mobility, or both, than that of BLV individuals as a whole.This could mean that they were more predisposed toward new aids.Whereas this might not affect mobility results or responses to DoUQ-MoB items of low abstractions levels too much, it might have yielded higher scores for those of high, such as [ Wanting to use both the tested aid and my current aid simultaneously is, likely].It is, however, difficult both to assess the impact of this, and mitigate it.
The most severe drawback of Parrot-VR is that ambient sounds in the VEs have not been implemented due to time constraints, and paired with them not walking physically, the participants' sense of orientation and relative distances were likely reduced.This shortcoming must be addressed in order to better gauge how the Audomni feedback affects macroscale navigation.Fortunately, assessing how the feedback affords object perception should not be as sensitive.This drawback also affected some questionnaire answers.As per one respondent: ''I could have told you if I had had background sounds when I tested this. . .so this question is irrelevant in today's test situation.'';however, seemingly most simply ignored, or tried to ignore, it.Another important consequence of the missing ambient noise was that the participants never experienced loud areas, and thus could only estimate such circumstances.
Other drawbacks include that, since Parrot-VR does not have a head-mounted display, participants with low-vision cannot use their functioning vision.Such a display would thus be beneficial, though would also make the system and VEs more complex, and could affect how users perceive the aid.Further, since the users did not wear or experienced a physical prototype, the DoUQ-MoB answers about physical properties are limited to an imagined physical ETA based on the IMU headband and cable and a supposed unit worn on the waist.

2) VIRTUAL ENVIRONMENT
A VE is perfectly reproducible, while it is still able to offer relevant large-scale urban scenes.Besides enabling identical environments regardless of testing location, as shown in this work, such reproducibility also allows for numerous interesting analyses.
Using the central station of Lund, Sweden, for the VE is unavoidably arbitrary.However, using a train station in general is deemed highly relevant as many BLV individuals encounter such frequently or incidentally.This also holds for the obstacles in the test scenes: construction sites and platforms, broken gutters, and truck loading platforms can all be considered arbitrary, but they represent general and relevant issues of BLV individuals, as shown by DoU-MoB; further, they are inspired by anecdotes from BLV individuals, and are unexpected obstacles that can appear even in known environments.It is highly likely that given another station and other, but still as relevant, obstacles would yield similar outcomes as those of this work.
Given that no participant was from Lund, and the specific route through the station, it was also improbable that familiarity with the station impacted any result.
A potential issue with the presented VE is its fidelity: all objects consist of the same material; there are no light sources; and, generally, features in the real environment smaller than 5 cm are not present in the VE.This means that there are no textures or colors present.Also e.g.cracks, small holes, small bumps, or gravel are not present.While this level of detail in the geometry should have negligible impact on the resulting feedback of the virtual Audomni (which is made to resemble the real one as closely as possible); various colors, textures, and materials could.The choice of detail level and material omission was made due to time concerns in the scene reproduction stage, required computational load of the VE, but also because simulating the performance of the real Audomni sensor in relation to different materials and light conditions would be quite an undertaking itself.
A final point is that, beside the user avatar, everything is perfectly still in the VE, including all objects representing pedestrians.This has advantages in increased inter-participant reproducibility and analysis, but it also makes the environments less realistic.For instance, it is likely that many real pedestrians would move out of the way, or offer to help, if a BLV user struggled to get past them for several minutes; on the other hand, participants never had to react to people moving inand-out of their FoV from the sides, or continuously moving in front of them.Thus, a tested device in this VE will in some regards seem more difficult than it would be in real-life, and in others easier.

3) DESIRE OF USE QUESTIONNAIRE FOR MOBILITY OF BLV INDIVIDUALS
Due to its focus on BLV mobility and mobility aids, its granularity, and that it is a PROM 1 , the DoUQ-MoB is situated well to address the severe lack of both user perception assessment, and low-abstraction-level measures, in the field.
Even though the DoUQ-MoB has been through validation with orientation and mobility experts and a pilot test with BLV respondents [19], these prompted a sizable number of modifications, and it is likely that further validation and revision would be beneficial.While the current scheme of answering should yield reliable intra-respondent comparisons, before-after comparisons using the DoUQ-MoB should be avoided until new validation rounds have shown acceptable test-retest stability.Further, a number of items regarding color and material were removed during this study, as it eventually became clear that most respondents struggled with them; how to adequately evaluate such has to be revisited.Next, even though the DoU itself is stated as a function between vectors of functionality and ergonomic complexity [20], the current DoUQ-MoB does not yet support aggregate scores of items; this could be made possible through a Rasch analysis, which could provide unidimensional constructs from them.One last improvement might be that, while the current administration time, predominantly a bit more than an hour, does not seem too troublesome (at least for a physical interview), shortening it can still reduce potential respondent fatigue.
Regarding language, despite all DoUQ-MoB items being presented in English in this work, these translations have only been tentatively made for illustrative purposes.Thus, it is not recommended to use an English version of the DoUQ-MoB before a proper translation-retranslation round from the Swedish originals has been conducted.
As noted in [10], most work in the field do not consider external conditions, e.g.lighting and weather, during evaluation.Indeed, in the DoUQ-MoB, efforts are made so that only one specific set of conditions are considered, as otherwise items would be to ambiguous, or a vast number of additional items would be needed.We do however agree that taking more conditions into account is important, so efforts should be made to accommodate them in some way.
The historical lack of user-reported results in the field, especially when comparing devices, provides little inspiration in how to present them.An exception is [37], where paired box plots are used.However, such might imply interval scales, while Likert-type and item-specific scales are ordinal.In addition, whereas box plots facilitate comparison of several items, for only pairwise comparisons histograms can be sufficient, as well as have the advantage of not obscuring the raw data.In any case, Figs.11 and 12 can be considered an exploratory attempt to visualize and analyze this type of results.
A last point regarding DoUQ-MoB is that this work is one of very few were a PROM has been applied to investigate the relation between current mobility aids and a proposed one-a practice which should be all-but-necessary to achieve an adequate aid that is accepted by blind and low-vision users.

4) TEST PROCEDURE
The short introduction to the Audomni feedback and VE, with a limited training time and only one traversal of each scene before the test sessions, can be considered both a feature and a weakness.It reduces the burden on-and allows for moreparticipants, and further, promising results can be deemed stronger as users should improve with experience.However, how functional the aid might be for an experienced user can only be theorized, and the tests might produce misleading results.Indeed, some participants were confused about the goal of a current scene, or if they should try to coastline to it or not.As such, potential issues found might not be issues with enough training, or in known areas.Still, as few incidences as possible should naturally be strived for, as it both removes such doubts, as well as decreases cognitive load and provides a higher DoU at the initial stages of trying an aid.
These circumstances are also suboptimal for the DoUQ-MoB.The respondents' ability to answer accurately will be affected to some degree with a total aid experience of only an hour (mean = 69 min), and only two traversals of each scene-with the second vastly different from the first.However, regarding testing and training duration itself, no participants seemed to have issues with it, nor showed any signs of degrading performance during testing.However, a few participants showed signs of fatigue during the later parts of the DoUQ-MoB afterwards.This would indicate that either the testing, training, or DoUQ-MoB might benefit from being shortened time-wise; or in any case should not be extended further.
Another issue is that only the proposed aid is tested in VR.It would be of great interest to do pairwise comparisons, where participants traversed the same scenes with the (or several) proposed aid(s), and also their current aid.Unfortunately, this would increase the test duration significantly, and whereas many ETAs should be relatively easy to integrate into Parrot-VR, the cane, and especially the dog, are not.
A solution for testing in known environments and pairwise testing with participants' currents aids, as well as for mitigating the potential VR drawbacks discussed in Sections IV-A1 and IV-B1, is to use participants' real-life everyday routes.This would also provide a greater sense of how an aid would function in their lives.Of course, this would reintroduce issues that VR alleviates, for instance with reproducibility and external conditions.Further the possibility to aggregate mobility results as herein would be diminished.Still it would yield maybe the most well-motivated test settings possible, and naturally any ETA that is to be realized into an end-product must be trialled in real-life settings eventually.The reproducibility issues could be controlled in part by reducing the allowable external conditions for tests; e.g.not performing tests in cases of rain (which in any case might be a requirement for an ETA prototype), snow, or excessive wind, and possibly only test in daylight.This could be a reasonable level for at least initial real-life tests, since the downside of such an approach naturally is that such conditions should be tested for at some point.When it comes to result aggregation in real-life tests, the DoUQ-MoB should be employable much like how it was for this work.An aggregate movement analysis would be decidedly more difficult; especially for participant-specific testing routes, any aggregate movement traces as in Figs.6-8 are not possible.However, collision and guiding breakdowns as in Figs. 9 and 10 could still be achievable, albeit without unique areas and obstacles.

V. CONCLUSION
Evaluating if potential end-users actually want to use a proposed mobility aid in a plain manner is unusual in the field.This work presents such evaluation, and shows a majority of BLV participants stating that they want to use Audomni.This claim is bolstered by the well-distributed and high number of BLV participants; that the employed DoUQ-MoB questionnaire was designed from best-known survey practices; and that the questionnaire considers the relation between the participants' current mobility aid and the tested aid-which also is seldom seen in ETA research.
Still, this work also presents a number of potential issues of Audomni that requires attention moving forward.
Regarding method, this is the first work employing the DoUQ-MoB and the Parrot-VR, which yielded numerous results of various abstraction levels through a movement analysis, the questionnaire items, and interviews.All these will guide the future design of Audomni, and would certainly benefit other ETA proposals as well.The major strengths of the test procedure are that it allows for a wide range and an adequate number of BLV participants; it evaluates an aid in relation to current aids; it exhibits great reproducibility, that also allows for aggregate results between participants; and it requires little preparation and time from the participants.

FIGURE 1 .
FIGURE 1. (Top) the first virtual training scene.(Bottom) the first virtual test scene which has added obstacles, hazards, and narrow passages.The object with a blue field protruding from it is the participant's virtual avatar, and it denotes the starting positioning of the scene as well as the field-of-view of the virtual electronic travel aid (ETA).The red dotted lines denote severe obstacles.The goal in these scenes is to reach ''Steps down 2'' by coastlining ''Platform edge''.Note the increase of obstacles and narrow passages in the testing scene, as well as that ''Gutter 3'' has been replaced with ''Broken gutter'', which extends to the middle of the platform.Whereas the regular gutters have a grating with 2 × 10 cm bars spaced 2 cm apart, the broken section has the same grating but with two bars missing at regular intervals of 44 cm.This represents holes of 10 × 10 cm, arguably enough to be a tripping hazard, and should thus be avoided.(The scenes have been slightly edited for visual clarity of this figure.)Figure and caption originally produced for this work, but published first with slight differences in[26].

FIGURE 2 .
FIGURE 2. (Top) the second virtual training scene.(Bottom) the second virtual test scene.The goal in the scenes is to reach ''Street curb'' by beelining, since there is no wall or edge to follow for approximately half the route.''Construction platform'' in the testing scene has a 5 cm thick horizontal plane at a heigh of 1.73 m.This height corresponds to the half-point between the avatar's eye height and total stature, and thus represents a head-height obstacle.Figure and caption originally produced for this work, but published first with slight differences in[26].
FIGURE 2. (Top) the second virtual training scene.(Bottom) the second virtual test scene.The goal in the scenes is to reach ''Street curb'' by beelining, since there is no wall or edge to follow for approximately half the route.''Construction platform'' in the testing scene has a 5 cm thick horizontal plane at a heigh of 1.73 m.This height corresponds to the half-point between the avatar's eye height and total stature, and thus represents a head-height obstacle.Figure and caption originally produced for this work, but published first with slight differences in[26].

FIGURE 3 .
FIGURE 3. (Top) the second virtual training scene.(Bottom) the second virtual test scene.The goal in the scenes is to reach the far edge of ''Street wall'' by coastlining ''Street curb'', and then face ''Street curb'' as if to cross the street.''Scooters'' in the testing scene extend approximately 5-20 cm from the ground and are tripping hazards.''Loading platform'' is 3 cm thick and situated 1.4 m above the ground, corresponding to chest height of the avatar; nevertheless, this is treated as a head-height obstacle throughout this work.Figure and caption originally produced for this work, but published first with slight differences in [26].
FIGURE 3. (Top) the second virtual training scene.(Bottom) the second virtual test scene.The goal in the scenes is to reach the far edge of ''Street wall'' by coastlining ''Street curb'', and then face ''Street curb'' as if to cross the street.''Scooters'' in the testing scene extend approximately 5-20 cm from the ground and are tripping hazards.''Loading platform'' is 3 cm thick and situated 1.4 m above the ground, corresponding to chest height of the avatar; nevertheless, this is treated as a head-height obstacle throughout this work.Figure and caption originally produced for this work, but published first with slight differences in [26].

FIGURE 4 .
FIGURE 4. Excerpt from and typical formatting of the DoUQ-MoB.Note that the questions are tentative translations of the Swedish questions.Also note the filter question 36, in which the respondent first choose response polarity, and then in question 37 choose the corresponding strength.Figure and caption reproduced with permission from J. Isaksson-Daun (manuscript to be submitted[19]), but published first in[26].
FIGURE 4. Excerpt from and typical formatting of the DoUQ-MoB.Note that the questions are tentative translations of the Swedish questions.Also note the filter question 36, in which the respondent first choose response polarity, and then in question 37 choose the corresponding strength.Figure and caption reproduced with permission from J. Isaksson-Daun (manuscript to be submitted[19]), but published first in[26].
FIGURE 5. Virtual reality system, Parrot-VR, test setup: 1) inertial measurement unit; 2) bone-conductive headphones; 3) controller; and 4) embedded computer.Auxiliary equipment: 5) audio interface; 6) laptop; 7) bone-conductive headphones; and 8) audio recording device.Note that the embedded computer hosts the software of both Parrot-VR and the Audomni ETA.Figure and caption reproduced with permission from J.Isaksson-Daun (manuscript to be submitted[19]), but published first with slight differences in[26].

FIGURE 7 .
FIGURE 7. a) Scene 2 movement trace of a theoretical participant walking in an ideal manner according to how the walking instructions for the participants were formulated.b) Scene 2 movement traces for all participants where they managed to walk on their own volition using the Audomni feedback.c) Scene 2 movement traces for all participants where they had to be guided by a test administrator.There were some issues in the ''Construction platform'' passage, and a considerable amount at the ''Bicycle group''-''Stroller group'' passage.

FIGURE 9 .
FIGURE 9. Condensed collisions breakdown, split from the left by type, obstacle and speed; and merged into reasons to the right.These are sorted based on: 1) feedback or method; 2) severe collision amount; and 3) collision amount.A severe collision is a collision at high speed, or with a ground-level change, head-height obstacle or low-level obstacle.One collision can have multiple reasons.The most prominent reasons due to the Audomni feedback are to the top right.For clarity, all obstacles with < 3 collisions are merged, and reasons with < 8 collisions are excluded.The boxes in the lower left corner depict how the collisions were distributed between participants, with each row representing collisions by a single participant Figure and caption originally produced for this work, but published first in [26].

FIGURE 10 .
FIGURE 10.Condensed ''guidings'' breakdown, split from the left by location and merged into reasons to the right.These are sorted based on: 1) feedback or method; 2) feedback guiding amount; and 3) method guiding amount.A guiding is any time, besides collisions, that a participant was intervened with.A guiding can have multiple reasons.The most prominent reasons due to the Audomni feedback are to the top right.For clarity, all locations with < 3 guidings are merged, and reasons with < 8 guidings are excluded.The boxes in the lower left corner depict how the guidings were distributed between participants, with each row representing guidings administered for a single participant Figure and caption originally produced for this work, but published first in [26].

TABLE 1 .
[26]icipant distribution.Table originally produced for this work, but published first with slight differences in[26].

TABLE 2 .
[26]ysis and example comments of the semi-structured interviews from DoUQ-MoB.Table originally produced for this work, but published first in[26].