Wearable AR and 3D Ultrasound: Towards a Novel Way to Guide Surgical Dissections

Nowadays, ultrasound (US) is increasingly being chosen as imaging modality for both diagnostic and interventional applications, owing to its positive characteristics in terms of safety, low footprint, and low cost. The combination of this imaging modality with wearable augmented reality (AR) systems, such as the head-mounted displays (HMD), comes forward as a breakthrough technological solution, as it allows for hands-free interaction with the augmented scene, which is an essential requirement for the execution of high-precision manual tasks, such as in surgery. What we propose in this study is the integration of an AR navigation system (HMD plus dedicated platform) with a 3D US imaging system to guide a dissection task that requires maintaining safety margins with respect to unexposed anatomical or pathological structures. For this purpose, a standard scalpel was sensorized to provide real-time feedback on the position of the instrument during the execution of the task. The accuracy of the system was quantitatively assessed with two different experimental studies: a targeting experiment, which revealed a median error of 2.53 mm in estimating the scalpel to target distance, and a preliminary user study simulating a dissection task that requires reaching a predefined distance to an occult lesion. The second experiment results showed that the system can be used to guide a dissection task with a mean accuracy of 0.65 mm, with a mean angular error between the ideal and actual cutting plane of 2.07°. The results encourage further studies to fully exploit the potential of wearable AR and intraoperative US imaging to accurately guide deep surgical tasks, such as to guide the excision of non-palpable breast tumors ensuring optimal margin clearance.


I. INTRODUCTION
Ultrasound (US) has become a mainstay both as a diagnostic modality and as an intraoperative imaging modality for image-guided interventions. The safe non-ionizing nature of this technology, combined with the low cost and high availability of the equipment, has enabled US to become the preferred imaging modality for a broad range of The associate editor coordinating the review of this manuscript and approving it for publication was Rajeswari Sundararajan .
interventional procedures, whenever a proper acoustic window is available. It is currently used on different levels ranging for example from an inside look before placing a pleural or ascitic drainage catheter to the guidance, monitoring, and follow-up in percutaneous thermal ablation of abdominal tumours [1] or the intra-operative guidance of lumpectomy in case of non-palpable breast cancer [2]. In the context of tumour resection, US offers exceptional visualization of the interface between healthy and diseased tissue, as highlighted in [3]. However, the simultaneous manipulation of the US probe and the cutting device constitutes a major difficulty for the surgeon, as it requires carefully coordinated relative handling. Moreover, the US images are typically visualized separately from the patient, on a conventional 2D standup display. Consequently, in addition to the complexity of always having to keep the cutting instrument and the target within the same US slice, a good sense of spatial relationships is required to mentally superimpose the 2D US image over the patient, and considerable hand-eye coordination for proper targeting of the lesion and alignment of the tool with the US 2D slice.
To overcome these issues in US guidance, different studies have already been presented in the literature, that involve the use of robotic systems to almost replace the clinician in the manual execution of the procedure, or the use of augmented reality (AR) technologies as an aid in the execution guidance. A first example of ultrasound-guided robotic intervention is presented in [3]. The study shows the feasibility of accurate autonomous US-guided dissections in a simplified invitro setup, with a reduced dimensionality (2D) setting. The authors proposed a new multimodal visual servoing approach that allowed the end-effector of the robotic system to perform the dissections task complying with the planned margins. Another example of US-guided robot-assisted intervention is presented in [4]. In this paper, the robot is used to automate a breast biopsy procedure. In particular, the robot has an actuated needle guide that directs the needle to a specified target within the US plane, but the actual needle insertion is left for the clinician to perform. The radiologist was able to perform the biopsy task on a phantom with sufficient needle placement accuracy, yet remaining in pseudo control of the procedure.
Other solutions found in literature alternative to robotics, are represented by navigation systems based on AR to improve the users' performances by means of a headmounted display (HMD) for US-guided puncture [5]- [8]. These studies employ HMDs (either VST, i.e. Video See-Through [5]- [7] or OST, i.e. Optical See-Through [8]) to guide a biopsy task, superimposing data derived from a 2D US probe on the patient's anatomy in a 1:1 ratio. AR HMDs prove to be particularly useful in tackling the hand-eye coordination problem, as they provide the user with an egocentric point of view over the working area, and the enhancement of the real scene with additional virtual content. In other terms, they provide the clinician with a view of the US image, or useful content extracted from it, directly overlaid and spatially registered over the patient anatomy. The hand-eye coordination process is thus simplified as ''the operator can point the needle tip directly into the ultrasound image'' [8]. These solutions are less expensive, smaller in terms of footprint, and they almost completely preserve the standard surgical workflow. However, manual dexterity and spatial coordination are still required to keep the needle visible in the 2D US image while inserting the instrument into the anatomy, since the US probe acquires just a 2D slice of the space containing (or not) both the target anatomy and the needle tip. The user has to handle the US probe to visualize the target anatomy in the 2D image and then to move the needle in the volume to lead the needle tip inside the target. To overcome this issue, a 3D US probe could be used. In fact, the employment of information extracted from a volumetric acquisition allows the clinician to visualise both the target lesion and the surgical instrument simultaneously, without having to move the probe. The authors in [9] presented a system based on the integration of a custom-made VST AR visor with a 3D US imaging system that, allowing the acquisition of an entire 3D volume, offers a three-dimensional vision of the anatomical area of interest and relates the needle to the entire acquired volume. The system shows a 3D reconstruction of a target lesion (extracted from the 3D ultrasound volume) superimposed on the anatomy encompassing it, and a couple of viewfinders aligned over a planned trajectory that intersects the target, to guide the insertion of the needle. The system offers the AR guidance for the insertion along the planned trajectory, whereas the depth is still controlled by the user looking in real-time at the US image or measuring the portion of the needle inserted with a caliper. The user study showed that the 80% of participants were able to perform the biopsy task on a 5 mm lesion, proving the accuracy of the proposed system of 2.5 mm. This approach, based on the viewfinders, works properly even in the absence of real-time visual feedback of the actual pose and depth reached by the instrument, as the biopsy need is to insert the needle along the right trajectory. The real-time movements performed to reach the right trajectory are not important, as it is sufficient to correctly align the instrument along the planned direction and to proceed with the insertion (guided by means of the viewfinders) up to the target.
In this new work, we investigate whether the integration of 3D US systems and AR HMDs can offer both the accuracy and the AR guidance information to guide a surgical dissection. Since the dissected tissues are determined by the real-time pose of the surgical scalpel tip, we integrated its tracking to offer immediate feedback of the instrument position/depth, which can thus be shown as a virtual widget within the augmented scene. In particular, in the case of the excision of non-palpable breast cancers, it is fundamental to dissect the tissues to approach unexposed targets while maintaining a safety margin. We evaluated with an appropriate user study the accuracy with which our integrated system can provide the position of a tracked scalpel. Therefore, we designed a second experimental setup to assess the efficacy of the proposed solution in guiding a dissection on a tailor-made phantom.

A. HARDWARE AND SOFTWARE COMPONENTS
The AR guidance system consists of three major components: a 3D US imaging system, a graphics and computational platform, and a Head-Mounted Display (HMD). Further details of the system components are provided in the following sections. VOLUME 9, 2021 A new concept hybrid video-optical see-through HMD is used within this work. It was developed in the framework of the European project VOSTARS [10] (Video and Optical See-Through Augmented Reality Surgical Systems, Project ID: 731974). The uniqueness of this new device with the associated software platform lies in its ability to provide both AR see-through mechanisms (Video See-Through, VST, and Optical See-Through, OST) within the same visor. Under VST modality, it can reach up to sub-millimetric AR registration [11], while in OST it can offer the user the comfort of an unaided natural vision. This goal was accomplished with the re-engineering of a commercially available OST HMD (ARS.30 by Trivisio [12]), as described in previous works. References [13], [14]. The OST-VST switching mechanism is obtained through a pair of liquid-crystal (LC) optical shutters, the transparency of which can be changed electronically via a voltage dimmer. The two LC panels are placed in front of the dual SXGA OLED displays of the ARS.30, which feature a 1280 × 1024 resolution at 60 Hz refresh rate, and a diagonal field of view (FOV) of 30 • . The inside-out tracking and the camera-mediated view of the VST paradigm are provided by a pair of front-facing RGB cameras rigidly incorporated in the 3D printed plastic shell that holds the headset and the two LC shutters. The stereo camera pair is composed of two LI-OV4689 cameras by Leopard Imaging, equipped with 1/3'' OmniVision CMOS 4M pixels sensor and M12 lens support. Both cameras mount a 6 mm focal length lens, which was chosen to restore a 1:1 scale factor at arms distance (∼40 cm) and they are set with a configuration of 2560 × 720@60 fps. To ensure a quasi-orthostereoscopic perception of the real scene under the VST modality, the cameras are mounted with an anthropometric interaxial distance (∼ 6.3 cm) and a fixed convergence angle of 3.4 • .
The visor has already been successfully used under VST modality to perform complex 3D trajectory tracing tasks on synthetic replicas of bony anatomies [15], and to guide in-depth high-precision manual tasks, such as biopsy intervention, on tailor-made phantoms [9].

2) AR SOFTWARE FRAMEWORK
The AR software framework is conceived for running AR applications for surgical guidance by supporting in-situ visualization of medical imaging data and it is specifically suited for AR stereoscopic headsets, both commercial and custommade. As comprehensively described in a previous work [14], the software can provide both optical and video see-throughbased augmentation of the real scene by using the VTK library, an open-source library for 3D computer graphics, modelling, and volume rendering. The AR platform is based on Compute Unified Device Architecture (CUDA), a multithread architecture that allows significant computational efficiency and ensures high flexibility both in terms of renderable contents and tracking capabilities. Concerning this latter feature, the software deploys a highly efficient inside-out optical tracking algorithm based on OpenCV API 3.4.1. The ad hoc algorithm performs the stereo localization of a triple of spherical markers and exploits this information to spatially register the digital content onto the reality before rendering it all on the displays.
In this work, the AR software framework runs on a Laptop PC with the following specifications: an Intel Core i7-8750H CPU @ 2.20 GHz with 12 cores and 16 GB RAM (Intel Corp., Santa Clara, CA, USA) as central processing unit, and a Nvidia GeForce RTX 2060 (6GB) with 1920 CUDA Cores (Nvidia Corp., Santa Clara, CA, USA) as graphic processing unit. Figure 1 shows the custom-made hybrid HMD and the laptop running the AR platform.

3) US ACQUISITION SYSTEM
The 3D US acquisition system used in this work is a Philips iU22 [16] (Philips Medical Systems, N.A.; Bothell, WA), combined with a VL 13-5, a 5 -13 MHz linear transducer. This system yields as output an externally processable DICOM volume of approximately 38 mm x 91 mm x 27 mm along the x-axis, y-axis, and z-axis, respectively.

B. US SYSTEM AND AR PLATFORM INTEGRATION
To determine in real-time the position of the tip of the scalpel with respect to the lesion to be removed, and thus to be able to correctly guide the dissection through AR guidance, several elements must be taken into account. Figure 2 shows all the elements involved, their respective reference systems (explained in the next sections), and the rigid transformation matrices relating them.
The following sections explain in detail the calibration procedures implemented to integrate all the elements and get the system up and running.

1) US SYSTEM AND TARGET LESION CALIBRATION FOR INTEGRATION WITH THE AR HMD
The augmentation of the real scene was obtained by superimposing the 3D medical information, provided by the FIGURE 2. Diagram of the component of the integrated system. The diagram shows the respective reference systems and the associated rototranslation matrices. In particular, WRS is the World Reference System, which is the global reference system associated with the visor; GFRS is the Green Frame Reference System, which is the reference system associated with the ultrasound probe; PRS, the Probe Reference System, is the reference system associated with the ultrasound acquisition volume; LRS is the target Lesion Reference System; RFRS is the Red Frame Reference System, which is the reference systems associated with the scalpel optical frame; TRS is the Tip Reference System, which is the reference system associated with the scalpel tip. In addition, T G is the instantaneous transformation matrix between WRS and GFRS, and T R is the instantaneous transformation matrix between WRS and RFRS, both obtained as the output of the optical tracking algorithm. X matrix defines the rigid transformation between GFRS and PRS, in the same way as Y defines the rigid transformation between RFRS and TRS. Lastly, W is the transformation matrix relating PRS and LRS, obtained through the segmentation process of the ultrasound volume.
volumetric acquisition of the US system, on the scene grabbed by the cameras of the HMD.
To ensure the proper locational realism, thus in order for the virtual content to be correctly spatially registered over the real scene, it was essential to know at all times the relative pose between the US probe volume and the AR HMD reference system. The inside-out tracking offered by the AR software framework presented in section II-A2 was used for this purpose. The design and 3D print of an optical frame was rigidly anchored to the probe so that the tracking algorithm could provide the instantaneous transformation matrix T G between the reference system associated with the US probe (GFRS in Figure 2) and the global reference system associated with the visor (WRS in Figure 2). The probe optical frame, shown in Figure 3, was properly designed with a software CAD (Creo Parametrics 6.0, [17]) and manufactured in acrylonitrile butadiene styrene (ABS) with a 3D printer. The three spherical markers were dyed in fluorescent green to improve the robustness of the RGB tracking.
To finalise the integration between the 3D US probe and the AR HMD, the definition of the transformation matrix X binding GFRS and the reference system of the US acquisition volume (PRS in Figure 2) is required. To determine this matrix, a known sized object with easily recognisable salient points (hereafter referred to as comb) shown in Figure 4 was acquired with the US probe in multiple steady poses, and the X matrix was determined offline with the routine implementing the hand-eye calibration method by Park and Martin [18], as thoroughly described in [9]. In particular, the comb was dipped in a water bath at 37 • , to best simulate an US real use condition, being this the average body temperature. A specific MATLAB (R2018b MathWorks, Inc., Natick, Massachusetts, US) routine was created and used to implement the registration between the cloud of twelve points acquired with the US probe and the actual CAD coordinates of the comb vertices.
The last transformation matrix needed is the one relating the local reference system associated to the target lesion (LRS in Figure 2) and PRS (transformation W in Figure 2). This can be easily derived offline from the acquired US volume, via a segmentation process. In conclusion, the final output of this procedure is the spatial relationship between the target, may it be the comb used for the calibration or the lesion, and the world reference system (WRS), and therefore the AR visor.

2) SCALPEL CALIBRATION
In order to properly refer the position of the tip of the scalpel with respect to the target lesion, the scalpel was sensorized VOLUME 9, 2021 FIGURE 4. The phantom designed for the hand-eye calibration between the probe reference system and the probe optical frame reference system. so that it could be tracked by the RGB cameras of the visor. A second optical frame was thus designed with Creo Parametrics 6.0, 3D printed and rigidly anchored to the scalpel in an ergonomic position for the instrument handling. Figure 5 shows the scalpel with its optical frame.
A further calibration step was required to determine the rigid relation (transformation Y in Figure 2) between the tip of the scalpel (TRS in Figure 2) and the reference system associated with the scalpel optical frame (RFRS in Figure 2). The EinScan-SE 3D Scanner by Shining 3D (SHINING 3D Tech. Co., Ltd. Hangzhou, China) was used to obtain a highly reliable three-dimensional scanning of the instrument. The stereolithographic (STL) file resulting from the scan was then imported into Creo Parametrics 6.0, in which the position of the scalpel tip relative to RFRS could be derived. The 3D scanning was chosen over other calibration methods, such as pivotal calibration [19], as considered to be less prone to errors. The pivotal calibration would have been affected by an intrinsic error due to the optical tracking that the 3D scanner allowed to avoid. On the other hand, the single-shot accuracy declared by the scanner manufacturer is ≤ 0.1 mm [20].
In summary, the pre-procedural integration steps can be listed as: 1) Design and 3D print the two optical frames to be rigidly anchored to the US probe and to the scalpel. 2) Compute the rigid transformation matrix X binding GFRS and PRS (i.e., US system calibration). 3) Compute the rigid transformation matrix Y between TRS and RFRS (i.e., scalpel calibration). 4) Compute the rigid transformation matrix W relating PRS and LRS (i.e., registration of the target lesion with respect to the US volume). 5) Generate the virtual content by processing the volumetric data from the US system and adding the planning (task-oriented) visual information. Whereas, at run-time, the relative poses T G and T R are determined by exploiting the inside-out tracking mechanism offered by the AR software framework presented in section II-A2.

C. VALIDATION STUDY
Two validation studies were performed to evaluate the accuracy provided by the entire system. More in detail: • a targeting experiment to assess the overall accuracy in estimating the relative distance between the scalpel tip and the US target as a result of the calibration procedures; • a preliminary phantom experiment to estimate the performance of the system in guiding a dissection task. The following sections describe the experimental protocols carried out.

1) SCALPEL-TO-TARGET DISTANCE ACCURACY -EXPERIMENT 1
The overall accuracy in estimating the scalpel-to-target distance was measured with a targeting task. The task consisted in reaching known points in space with the tip of the scalpel and calculating the Euclidean distance between the position of the tip in PRS, reported as output by the system, and the actual coordinates of the known points again referred to PRS. The known points belong to the same comb used for the calibration process. Owing to obstruction issues associated with the geometry of the comb, only nine of the twelve  The nine vertices used are identified in Figure 6. To determine the coordinates of the nine comb vertices in the PRS, the phantom was dipped once again in a water bath at 37 • , and acquired with the US probe. Once the US volume was acquired, the water in the tray was drained to ease the scalpel movements.
The AR HMD was anchored to a flexible mounting arm and moved in three different positions to simulate plausible user's points of view in static conditions. For each pose of the AR HMD, it had to frame both the probe optical markers and the whole comb within the field of view of the two RGB cameras, to guarantee the tracking of both the probe and the scalpel for all the nine comb vertices. Figure 7 presents the experimental setup designed. The scalpel was anchored to a second mounting arm. For each pose of the visor, the scalpel was maneuvered to make its tip touch each of the comb vertices three times. Once the scalpel was correctly positioned, the arm was fixed and the coordinates of the scalpel tip with respect to the PRS were acquired in real-time for 5 seconds. The coordinates acquisition was extended over time in order to average them and make the measurements less prone to errors introduced by the optical tracking. Between each tapping on one vertex, the scalpel was moved away and then re-positioned on the tip of the comb, to minimise as much as possible the error introduced by the user's skill in precisely positioning the scalpel, thus averaging the placements over three distinct measurements.
The acquired set of data (81 scalpel positioning, given by 3 placements x 9 vertices of the comb x 3 poses of the visor) was processed in MATLAB, and the Euclidean distances between the real coordinates of the vertices (obtained from the volumetric US acquisition) and the coordinates of the tip of the scalpel in PRS (resulting from the mediation of the system output on the three acquisition taken) were calculated.

2) FABRICATION OF THE ULTRASOUND PHANTOM
A synthetic US phantom was used for testing under realistic conditions the feasibility of providing AR information when performing a dissection. In general terms, dissection is performed to reach target structures deep within the body and not immediately reachable underneath the surface. Thus, to realistically simulate this task, a custom-made US phantom with a target lesion inside was designed and manufactured.
The design and manufacturing of the US phantom included the following steps, described in detail in this section: 1) Study of the literature for selecting the best material for replicating both the speed of sound propagation and the mechanical properties of the natural tissue. 2) Design/selection of moulds for casting the selected material. 3) Definition of a recipe for the fabrication of simulated lesions. 4) Definition of a recipe for the fabrication of simulated parenchyma. 5) Definition of a strategy to include the simulated lesion in the parenchyma. In general terms, tissue-mimicking materials should be designed to best replicate the physical, mechanical, and chemical properties of the human tissue they simulate, VOLUME 9, 2021 targeting the specific medical imaging modality required. In particular, for the US imaging technology, tissuemimicking materials are usually created with similar acoustic properties (i.e., acoustic attenuation, acoustic impedance, and speed of sound) to those of the soft tissue of interest [21]- [24]. Several materials at the state of art mimic these mechanical and medical imaging properties. These can be classified into biopolymers and chemically synthesised polymers. The primary advantage of biopolymers, such as agar, agarose, gelatine, and gellan gum, lies in their similarity to human soft biological tissues, owing to the high mass fraction of water they contain (> 80%) [25]. However, due to water evaporation and especially bacterial growth, biopolymers, as well as organic tissue such as chicken or turkey, are not recommended for long-term use or storage [26]. Conversely, chemically synthesized polymers such as polyvinyl alcohol (PVA), polyvinyl chloride (PVC), Room-Temperature-Vulcanizing (RTV) silicones, polydimethylsiloxane (PDMS), and polyurethane (PU) are more stable and durable over time [27]- [29], but the lack of water makes them slightly different from human tissues. Of these materials, PVC most closely matches the properties required of an US phantom. PVC-Plastisol (PVC-P), which is a suspension of PVC in a liquid plasticizer, has advantages both in terms of durability (it resists bacterial attack and moisture loss compared to biopolymers) and acoustics. Indeed, speed of sound in this material (∼1400 m/s) [28], [30], [31] is closer to that of generic human soft tissue (∼1500-1600 m/s) [32]- [35] compared to silicones (∼ 1000 m/s) [36], [37] or PU (∼1800 m/s) [38], [39]. As for the mechanical properties, elastic modulus and hardness of PVC-P are closer to soft tissue than PDMS and PU, an essential factor for the phantom to have a tactile feeling similar to real tissue. In addition, PVC-P is easier to manufacture than PVA and allows tailoring the phantoms with different degrees of hardness and workability by simply changing the ratio of its additives (softener/hardener, dyes, etc.). For all these positive characteristics, PVC-P was the material chosen in this work for the realization of the US phantom.
As for casting moulds, a two-part mould generating four spherical lesions of 10 mm each was designed with a Creo Parametrics 6.0 and manufactured in ABS (Figure 8a). Moreover, a wooden cube-shaped box, measuring 8 × 10x8 cm, was used as a mould for the parenchyma to obtain a nonanthropomorphic parallelepiped-shaped phantom.
The operative procedure for manufacturing the lesion and the parenchyma is reported below together with a description of the method for incorporating the lesion within the simulated parenchyma.
Cooked non-doped PVC-P has an anechoic response and is translucent in appearance. To enhance the distinction of the lesions from the surrounding parenchyma, and therefore to determine at a glance if the scalpel had excessively cut reaching the lesion, pure liquid PVC-P (from the M-F Manufacturing Co., Fort Worth, TX, USA) was mixed in a pot with white dye (Smooth-On Inc., Macungie, PA, USA) and placed in a vacuum chamber to remove air bubbles. Afterward, the pot was placed on a hotplate under a fume hood (Pratika Ferraro, 1AKS0000-ST-MA-L1200 model) to slowly heat the material. PVC-P must be heated under a fume hood to minimize inhalation of vapor, which may cause minor irritation to skin or eyes when exposed to excessive amounts.
Owing to the small quantities of material required to produce the lesions, it was not necessary to stir the material as the heat could evenly distribute throughout the pot. Once the material reached a temperature of 180 • , it was injected into the printed mould through a syringe and left to cool under the fume hood (Figure 8b).
For the manufacturing of the parenchyma, PVC-P was instead doped with 2% by weight with graphite, to increase the speed of sound of the material up to 1490 m/s. Graphite was added through a sieve before heating the material, to prevent lumps formations. The material was dyed in green (Smooth-On Inc. dye) to improve the contrast between the scalpel and the surface of the phantom (PVC-P doped with graphite normally takes a jet black shade). Once again, the mixture was placed on the hotplate under the fume hood and slowly heated up to the temperature of 180 • . Given the large quantity of material and to prevent the graphite from settling at the bottom of the container, the material was continuously stirred throughout the heating time. Finally, the mixture was cast into the wooden mould and a lesion was placed at a randomly chosen depth from the entry surface of the phantom by suspending it inside the mould with a copper wire (wire thickness 0.112 mm). Before placing it in the mould, the lesion was kept in the freezer for 3 hours to cool it and thus prevent the hot PVC-P of the parenchyma pouring from melting the outer surface of the lesion and deforming it or changing its size. Figure 9A displays the resulting phantom, whilst Figure 9B shows the included lesion at the US view.

3) PRELIMINARY USER STUDY -EXPERIMENT 2
The level of accuracy of the integrated system in estimating the depth of the tip of the scalpel, and thus guiding a dissection task was checked. For this purpose, three participants, recruited among university staff and faculty members, were asked to wear the AR HMD and to perform a dissection with the sensorized scalpel reaching a predefined distance to an occult lesion Table 1 reports the participants' demographics, whereas Figure 10 shows the experimental setup while one of the participants performs the task.
To create the virtual content to augment the real scene acquired by the RGB cameras, a volume containing the target anatomical structure was acquired with the US probe, and the lesion was extracted via a dedicated segmentation software (ITK SNAP, [40]). A Virtual Reality Modeling Language (VRML) file was created from the segmented mesh and uploaded onto the AR platform so that it could be displayed in the visor coherently registered over the real phantom. Though the lesion was not the real target of the experiment, it was decided to still show it in AR to guide the incision to be medial with respect to the target.
Hereby, to guide the in-depth incision, the target lesion and two guiding trajectories were virtually added to the real scene and could be seen through the visor. The two guiding trajectories, 2.5 cm in length each, were placed one on the surface of the phantom (hereafter defined incision trajectory) and one (hereafter referred to as alignment trajectory) at about 13 cm away from it, approximately the scalpel length, aligned over the ideal incision trajectory. In a first instance, only the lesion and the incision trajectory on the surface were shown ( Figure 11A). The participants were asked to line up their monocular point of view so that the incision trajectory was centered with the lesion, and to place the tip of the scalpel over on the incision line. Subsequently, the alignment trajectory was shown. The user had to align the bottom of the scalpel with this second line, so as to be sure that the deep incision was made along the right trajectory ( Figure 11B). The application gave also a real-time indication of the depth of the scalpel tip relative to the surface of the lesion. To determine this depth, the position of the scalpel tip was projected over the axis of the ideal entry trajectory, and the distance along this axis between said point and the surface of the lesion was calculated (Figure 12).   figure) between said point and the surface of the lesion along this axis is computed. The angle θ between the ideal insertion plan and the one actually followed by the scalpel was also measured but not shown in the visor's display.
The information about the depth was shown in AR in the up left corner of the visor's display. It was decided to leave a safety margin from the lesion, as it usually happens in surgical intervention for the removal of malignant formations  TABLE 3. Performance of the system in guiding a dissection task. Results are expressed in terms of relative depth to the target and inclination of the actual trajectory relative to the ideal one.
(e.g, wide excision of breast lesions). The established margin was 5 mm, hence the users were asked to stop when the indicator in the visor marked a depth of -5 mm. Table 2 reports the results of the accuracy tests for the estimation of the scalpel-to-target distance. For each pose of the visor, the error over all nine vertices of the comb was evaluated and the results are reported in terms of maximum value, median and interquartile range (IQR). General metrics were also evaluated, including all data acquired on all vertices over all three poses. Overall, the median value of the error was 2.53 mm, with an IQR of 1.86 mm, and a maximum error of 4.45 mm.

B. PRELIMINARY USER STUDY RESULTS
The actual distance between the tip of the scalpel and the surface of the lesion was measured to accurately assess the user performance in the AR-guided dissection tasks. More specifically, a volumetric US scan was acquired at the end of each trial. The lesion and the scalpel were segmented with ITK SNAP, and the respective meshes were then imported into Creo and registered over the pre-interventional acquisition of the lesion. The two linear trajectories were also imported within the same file, to determine whether the scalpel was driven into the phantom following the correct trajectory or not. Distance and Angle Creo features were used to determine respectively the distance between the scalpel tip and the surface of the lesion (computed as the distance from the scalpel tip to the lesion centre minus the radius of the lesion), and the angle between the ideal insertion plan and the one followed by the scalpel. The results for each participant are reported in Table 3.
Overall, the average distance at which the scalpel was brought with respect to the surface of the lesion was 4.4 mm, resulting in an error of 0.6 mm with respect to the planned distance, whilst the average error of the inclination of the trajectory was 2.07 • .

IV. DISCUSSION AND CONCLUSION
In recent times, US imaging is emerging as both diagnostic and interventional tool due to its non-ionizing nature, the small footprint and high availability of the associated equipment, and its cost-effectiveness compared to other imaging technologies. Similarly, the use of AR HMD in the surgical field is becoming more and more widespread and can span from the monitoring of patients during surgical procedures [41], [42] to the actual navigation of the surgery [43], [44]. Within the latter context, what we propose in this work is an integrated system composed of a 3D US imaging system and a HMD with its related computational AR platform, for the guidance of a deep dissection task. The peculiarity of our system is the tracking of the surgical instrument used for cutting (a standard scalpel in this case, but it could be an electric scalpel or any other surgical instrument) so that immediate feedback on the position of the scalpel could be provided to the user.
We designed and performed an experimental study to evaluate the overall accuracy of the system in estimating the scalpel-to-target distance. The overall error, with a median value of 2.53 mm and an interquartile range of 1.86 mm. It is important to emphasize that whereas in the previous work the error component related to the optical tracking was due to the detection of a single object, in this study the objects to be tracked simultaneously were two. This led to an inevitable but not substantial error increase, demonstrating the reliability of the tracking algorithm implemented. We confidently claim that the targeting accuracy achieved makes our system recommendable for surgical applications on targets of at least 5 mm in diameter. To the best of our knowledge, there are no previous works in literature that exploit an integrated system like ours (3D ultrasound imaging system plus wearable AR visor) to guide dissection tasks. However, we can assert that the median accuracy of our system stays within the range of alternative mechatronic and robotic-assisted US-guided breast biopsy systems proposed in recent years, with reported system errors of approximately 1.0 to 3.0 mm [45].
A preliminary user study was designed to test the feasibility and the accuracy of the system in providing the entry depth of the scalpel tip relatively to a target during the execution of a dissection task. To this end, we engineered an ad hoc US phantom with synthetic lesion embodied in it, simulating a possible target to approach but not to incise during the execution of a dissection (may it be a vessel to be avoided or a tumour that is to be extracted). Participants were asked to perform the dissection and to stop when they visualized indicated in the visor a distance of 5 mm between the tip of the scalpel and the surface of the lesion. Despite their inexperience in performing this surgical task, all the participants were able to carry out the dissection along the indicated trajectory without ever reaching the target lesion. In particular, the average error made, evaluated as the distance between the scalpel tip and the centre of the lesion minus the lesion radius, was 0.6 mm. As for the inclination of the scalpel, the angular error between the theoretical cutting plane and plane actually followed by the scalpel was also checked. The average error measured was 2.07 • , demonstrating that the visualization method used for cutting guidance (the two trajectories to align tip and bottom of the scalpel) proved to be reliable.
One limitation of this work is due to the bulkiness of the optical frame placed on the scalpel. Although it is placed in an ergonomic position for the grip of the instrument, the frame has still a not negligible encumbrance, which can limit the possible movements of the scalpel, especially if these are made in the proximity of the US probe. The authors' future endeavor is the design of a more suitable optical frame that will allow for better instrument maneuverability. The future idea is to sensorize other surgical instruments as well, so as to be able to guide other surgical tasks in addition to dissections. For instance, in the case of a wide excision intervention of breast cancer, it would be useful to have not only the regular scalpel traced, to guide the preliminary dissection to approach the tumour, but also the electrical scalpel, which is the one actually used to accomplish the excision.
A further limitation, already observed in the previous work, has a technological nature. To date, the real-time video streaming of a volumetric acquisition is not yet allowed by the US imaging systems. The data acquired can only be stored on a hard disk and exported for external processing. On this basis, we were forced to use a pre-operative acquisition of the lesion, and at the moment we are not able to update in real-time the pose of the virtual 3D model of the lesion with respect to the probe.
Despite these limitations, the proposed integrated system has proven to be very effective and accurate in guiding not only the dissection task but also in providing an indication about the position of the scalpel. The surgeon could thus exploit the system to perform a whole tumour excision with the confidence of leaving sufficient margins of healthy tissue around the tumour.
NADIA CATTARI received the M.Sc. degree in biomedical engineering from the University of Pisa, where she is currently pursuing the Ph.D. degree in clinical and translational science with the EndoCAS Centre. Her research interests include vision augmentation, stereoscopic 3-D displays, computer graphics, analysis of perceptual issues in the interaction with augmented environments, and calibration in augmented reality. She is involved in research regarding the integration between augmented reality platform and various medical imaging modalities and the efficacy of wearable augmented reality head-mounted displays as tools for surgical guidance and/or surgical training in various types of procedures.
SARA CONDINO received the Ph.D. degree from the University of Pisa, Italy, in 2012, with a thesis regarding the development of an electromagnetic navigation system for endovascular surgery and new strategies for innovative medical devices testing. Her current research is carried out in the context of computer-assisted surgery and in the development of innovative technologies based on virtual reality and physical and hybrid simulation for rehabilitation specialists. She is the author of scientific articles on ISI journals and patents. VINCENZO FERRARI (Member, IEEE) received the Ph.D. degree from the University of Pisa. He is currently an Assistant Professor of biomedical engineering at the Department of Information Engineering, University of Pisa. He is the Coordinator of the EndoCAS Centre, University of Pisa. He is involved in several national and international research projects. He is the author of more than 80 peer-reviewed publications and has five patents. His research interests include image-guided surgery and simulation, computer vision, and augmented reality devices and applications in medicine. VOLUME 9, 2021