Scheduled System Maintenance
On Friday, October 20, IEEE Xplore will be unavailable from 9:00 PM-midnight ET. We apologize for the inconvenience.
Notice: There is currently an issue with the citation download feature. Learn more.

IEEE Quick Preview
  • Abstract



In the graphics representation of the reality-virtuality continuum by Milgram and Kishino [1], Augmented Reality (AR) is regarded as a way to “augment” the real world with virtual objects. While, the goal of Virtual Reality (VR) is to create entire virtual worlds, in an AR system virtual objects are generated in such a way that they appear to coexist with the real ones. Although AR may involve all the senses, virtual objects are frequently represented by computer-generated contents that needs to be presented to the user. In [2], Azuma et al. identified three properties characterizing an AR-based system: combination of real and virtual objects in a real environment, interactive execution and alinement of virtual objects with respect to the real ones.

Although AR technologies can be dated back to 1960s [3], it was only in 1990s that software and hardware AR technology found remarkable application domains. In 1997, Azuma presented a first survey on AR [4], where he identified a set of key applications. Manufacturing and repair appeared immediately as challenging domains. In 2001, the survey by Azuma was complemented by [2], where the impact of mobile devices as well as issues related to the social acceptance of AR were outlined. On 2004, Navab [5] identified a set of possible high-impact applications for industrial AR. Service and maintenance played again a key role. Moreover, the author outlined how three main issues had to be addressed by “useful” AR-based applications: reliability, user friendliness and scalability beyond simple prototypes.

The Technology Acceptance Model (TAM) proposed by Davis on 1989 [6] and its evolutions clearly outline how the perceived usefulness and the ease of use are at the basis for the acceptance and adoption of any new technology. For the spread and acceptance of AR systems, the turning point has been represented by the advancements in the field of mobile technology. AR is now quite well consolidated and a large number of applications for smarthphones and tablets support the users in a wide range of applications. Tourism, advertisement, shopping, and social networks are just a few domains where AR proved to be capable of bringing significant benefits. According to a study by ABI Research [7], the US market for AR is expected to hit $350m in 2014, up from about $6m in 2008. Furthermore, some recent advances in AR technology, such as the Google Glass project [8] and contact lenses by Innovega [9], are opening exciting scenarios [10].

Regardless of such new opportunities, maintenance, repair and assembly are still identified as strategic application fields, since the reduction of associated costs represents a key goal in many domains. Indeed, cost changes would depend on the particular application scenario. For instance, maintenance and repair costs make up just 4 percent of the ownership costs of a car [11]. However, in the aircraft domain, maintenance costs can reach 80 percent of the entire product lifecycle (from design to disposal) [12], and are expected to increase overall to $54 billion in 2015 [13]. Hence, any technological advancement is carefully considered in order to take the opportunity to reduce these costs.

Even though Interactive Electronic Technical Manuals (IETMs) are frequently used to replace paper-based instructions [14], they are not completely part of the interaction between the technician and the equipment to be maintained. Therefore, both the overall time of the repair/assembly task and the cognitive load on the maintainer might increase [15], [16]. This could also affect the number of errors introduced in the maintenance process.

Benefits of AR-based documentation for maintenance are well depicted in [17] and [18], where it is outlined how AR technology can reduce costs up to 25 percent and improve performances up to 30 percent. Moreover, although over the past decades AR-based applications for maintenance were used only by technical specialists, current solutions promise to profoundly change the way end-users will perform many of their daily tasks. For instance, traditional paper-based owner’s manuals for ordinary car maintenance, furniture building instructions and installation manuals of electrical appliances could be soon replaced by AR-based applications on mobile devices. The number of people possibly involved in using AR-based applications is potentially huge and the social impact cannot be neglected. Even though AR systems can provide meaningful advantages in maintenance, repair and assembly tasks, drawbacks might be also experienced. For instance, a detailed analysis of stress and strain produced by a long-term usage of AR technologies in presented in [19].

This manuscript aims to present trends, challenges, opportunities and threats of AR technology applied to the maintenance domain. Section 2 reviews the state of the art. Section 3 analyzes new and future trends by considering also other applications that may have an important social impact. Open problems and limitations of current solutions are addressed in Section 4, whereas the approach that is being pursued in the context of the European project EASE-Formula$\text{R}^{3}$ to tackle some of the current challenges is introduced in Section 5. In particular, an AR-based architecture exploiting reconfigurable procedures that can be modified during tele-assistance sessions to improve the effectiveness of existing maintenance processes with the goal of reducing times and costs is illustrated in Section 6. Finally, some preliminary experimental results obtained using the above framework are reported in Section 7.



The idea to convey maintenance instructions to technicians by using AR-based systems is not new. Early examples, which can be dated back to the 1990s, are well surveyed in [20] and [21].

AR-systems are usually classified based on different parameters, including tracking technology, human-system interaction method, data management strategy, etc. A rather common categorization, which is used in this paper, considers the visualization device. Predictably, a key role is played by wearable technologies [22]. In particular, since from the early experiments by Feiner et al. [23] with a head-worn AR prototype designed to support end-users in performing simple maintenance procedures on a laser printer, a number of head mounted display (HMD)-based solutions were developed.

For instance, in the Etälä project [24], tele-assistance and AR were exploited to establish a communication channel between maintainers and remote experts (in a so-called tele-maintenance framework [25]). Experts could navigate a virtual model of a real object and share useful information to support maintenance and repair operations. The same approach was adopted in [26], where VR and AR were used to remotely support technicians as well as trainees. The German Federal Ministry of Education and Research (BMBF) sponsored the ARVIKA project [27], which was aimed to the design and implementation of a head-worn AR-based user-centered tool to support the development, production, and servicing of complex technical products and systems. STARMATE [28] was one the first examples of multimodal augmentation, where virtual objects, textual hints and audio messages were used to guide and support the maintenance of mechanical parts. Collaborative frameworks have been also proposed. For instance, AR-technologies are used in [29] to train technicians in assembly tasks of complex systems such as aircraft engines.

Several studies analyzed the potential of such systems and their impact on training and maintenance processes, by considering different application scenarios. As a matter of example, the ARMAR project [15], [30], [31] considered advantages and drawbacks of various hardware and software solutions for maintenance job aiding in the military field. Ke et al. developed a prototype to maintain PCs [32]. The ARVIKA project addressed the automotive and aircraft industries. Results produced by ARVIKA represented the input for the ARTEAS project [33], which explored the use of AR technologies in more general contexts. Schwald and Laval focused on the usage of AR solutions for maintenance operations in generic industrial scenarios [34]. In [35] and, more recently, in [36], AR was used to simulate and validate the programs of Computer Numerical Control (CNC) machines. Efficiency of AR in the manufactory industry was specifically measured in [37], where performances of people involved in an assembly tasks have been measured in terms of time saved and errors avoided due to either wrong tool selection or wrong part positioning.

In some works, like in [38] and [39], AR is referred to as mobile to denote the possibility for the users to move, typically in an industrial site, in order to find objects to be maintained. For instance, the goal of the PLAMOS project [39] was to support owners and operators in plant maintenance and repair. Specifically, a marker-based solution was used to easily identify industrial facilities throughout the plant. Markers were used in a similar way in [40] to support factory planning in the automotive industry. In [41], an AR-based Operations and Maintenance Fieldwork Facilitator (AROMA-FF) was proposed, with the goal to reduce the time needed for the end-users to locate target objects.

Portable devices, such as smartphones and tablets, support a different AR paradigm. In fact, if HMDs enable see-through visualization, mobile devices implement the so-called handheld display paradigm. As reported in [42] and [43], Handheld Augmented Reality (HAR) has been fueled by the spread of personal devices and a lot of solutions are already available.

However, HAR applications in the maintenance domain are still quite scarce. An example is reported in Fig. 1, where the use of a smartphone for repairing a netbook is illustrated. The technician frames the object under maintenance using a smartphone (Fig. 1a). When the object is recognized, some virtual cues, e.g., a text message and a computer-generated animation, are aligned and displayed on the screen (Fig. 1b). A new configuration of the object has to be recognized at each step of the procedure, which evolves according to a pre-defined state diagram.

Figure 1
Fig. 1. Handheld Augmented Reality for maintenance operations: a) real object under maintenance framed by a smartphone, b) close look on the HAR application graphics interface.

By considering Fig. 1 it is immediately clear that at least one hand of the maintainer is used to hold the HAR device. Hence, is not possible to have both hands free to perform the task. Moreover, the observation of the real object through the device camera might involve safety issues. Despite possible limitations, some examples of HAR-based maintenance are known in the literature and a Fiatech’s webinar presented benefits coming from the use of AR applications on mobile devices [44]. Thus, for instance, Kahn et al. proposed a HAR solution able to support the overall lifecycle of construction and facility management [45]. AR-based training systems for maintenance applications using mobile devices are illustrated in [46] [47] [48]. Kim and Moon [49] focused on car maintenance and introduced a new and potentially high-impact field of HAR, i.e., the replacement of handbooks and paper-based instructions.



Although many applications of AR could get a higher social value, solutions of AR-based maintenance tailored to end-users, i.e., not for specialists, are quickly growing as they promise to have a significant influence on everyday life. In fact, any dematerialization process is expected to produce a social impact [50] and, sometimes, the traditional interaction procedures can be even toppled.

Car routine maintenance is one of the fields that is doomed to be profoundly changed by the introduction of AR-based solutions. Some car makers now provide their users the owner’s manual also as an application for mobile devices. For instance, it is already possible to recognize more than 300 individual components of the Audi A3 both on the instrument panel and under the hood. In this way, relevant how-to information and virtual overlays of maintenance procedures can be conveyed to the user directly onto his or her personal device. For example, after framing the engine with the device’s camera, the AR application could provide an animated overlay of virtual objects, with instructions on how to locate the engine coolant and refill it to the appropriate level [51]. Moreover, Metaio, a company leader in AR technology and solutions [52], recently announced the development of the first-ever hands-free marker-less augmented automotive manual using Google Glass.

Some assembly tasks could also take advantage of AR technology [53] [54] [55]. For instance, assembly of pretty simple objects, such as furniture items, could be effectively supported by AR solutions. Several applications for mobile devices already provide end-users tools for a virtual preview of customizable furniture in the environment, thus implementing an enhanced digital catalogue [56]. Other examples are represented by solutions developed by Mitsubishi Electric and NGRAIN 3D. In the first case, installation and maintenance processes of heating and cooling products are explained by an AR application. In the second case, an application for training on the maintenance of industrial parts is provided. Some companies also quantified benefits obtained by adopting AR. For instance, a toymaker (Lego) provided users a tool for enhancing the assembly experience and observed that sales increased by 15 percent after the introduction of the tool.

Examples above clearly show the potential impact of AR on daily activities in the maintenance, repair and assembly domains. Nonetheless, as it will be detailed in the following, there are still problems that need to be addressed.



This section aims to present technical problems pertaining the design and development of AR-based applications for supporting maintenance, repair and assembly tasks (other issues, e.g., about privacy, security and technology acceptance, are out of the scope of the present work). Indeed, the evolution reached by existing AR Software Development Kits (SDKs) and libraries such as Metaio [52], Vuforia [57], Artoolkit [58], etc. is so high that designers and developers can focus today on developing application logic and contents. However, it has been shown that there are two main issues, specifically pertaining the training required to setup the tracking mechanism and the reconfigurability of the overall system, that still require a special extra attention.

The first issue is related to the capability of AR applications to recognize objects in the real world and track their pose. Approaches based on the use of artificial elements, like markers, displaced in the environment (e.g., sticked to the objects to be tracked) are very robust and work fine. Training the system to work with such elements is also quite easy, since they are designed to be clearly distinguishable. However, it is not always possible to add such information. Hence, a tracking based on natural features, i.e., on object characteristics that are inherently available, is often desirable. In this case, the tracking system can be trained basically in two ways: by images, or by 3D CAD models.

With the first alternative, training is performed by collecting several pictures (referred to as training images) of the real object from different viewpoints. Training images are processed offline in order to identify and extract a certain number of so-called image descriptors. During operation, the above processing is repeated online on images gathered by the user device’s camera. When a match between offline and online-computed descriptors is found (a threshold is generally set to define recognition precision and robustness), the tracking step can be executed, thus correctly aligning virtual objects to the real ones. Unfortunately, this approach is strongly dependent on environmental conditions as well as on possible changes in the surface of the real object with respect to what is actually pictured in the training images. Different lighting, reflections, shadows, dust, dirt, rust, etc. may compromise both the recognition and the tracking of the object. A possible solution could consist in replacing training images with photorealistic renderings. Starting from 3D CAD models of real objects, it could be possible to simulate and render different environmental conditions as well as different settings.

However, a more effective alternative is to directly train the system using the 3D CAD models. In this case, the model is used together with a line representation of the model itself (a kind of silhouette-like drawing), which allows the tracking system both to recognize the object and to correctly align the camera to it. Fig. 2 shows a tracking system trained by 3D CAD models that is able to recognize an inkjet printer. A flat shaded and transparent representation of the model helps the user to align the camera with the real object. The user can adjust the number of polygons of the model to control tracking robustness and precision. The advantage of this second approach is that it is independent both of environmental conditions and of other influences that might modify object surface appearance. The drawbacks are related to the overhead due to the modeling phase of polygonal and line representations, which can be a serious constraint for the development of “home-made” applications.

Figure 2
Fig. 2. A representation of the 3D model helps the user to correctly align the device’s camera with the real object.

The second main issue is related to system reconfigurability. AR maintenance applications are usually designed to support a set of procedures that consist of a fixed number of well known steps. As discussed in the Section 3, the design of a state diagram is at the basis of any procedure. Each state corresponds to a well defined step of the procedure and system behavior needs to be clearly specified by means of state transitions. However, a dynamic reconfiguration of the state diagram that might be required, e.g., to deal with unexpected situations not contemplated in the existing procedure, is, in general, a very complex task that is hard to be performed at runtime.

The above issues are currently been addressed in the EASE-Formula$\text{R}^{3}$ project, whose goal is to provide maintainers a flexible and robust AR-based tool for managing reconfigurable procedures.


THE EASE-Formula$\text{R}^{3}$ PROJECT

The EASE-Formula$\text{R}^{3}$ project 1 aims to develop an integrated framework for a cost-effective and easy Repair, Renovation and Re-use of machine tools within modern factories. EASE-Formula$\text{R}^{3}$ will cover the entire lifecycle of the above framework, from the design stage through the operative life. The project, which started on July 2013, is co-funded by the European Commission under the FP7 Framework Programme and involves 14 partners from 7 European countries.

Machine tool makers represented in EASE-Formula$\text{R}^{3}$ expressed the need to provide their customers an efficient assistance. However, arranging customer services in different countries can be very expensive. Thus, technicians often travel from the headquarters to reach the customer’s site. Moreover, machine tools are often customized based on customer’s needs and cannot be considered mass production objects. Custom objects can require custom procedures, which are very difficult to standardize and define.

In the scenario depicted above, AR solutions are expected to be used for remotely supporting technicians, by means of a tele-assistance approach, in repair and maintenance tasks for broaching, milling and stone cutting machine tools. Despite the vertical domain, the project aims to develop a flexible methodology to allow experts at the headquarters to dynamically reconfigure/modify AR procedures to be implemented by technicians at the customer’s site.

In a preliminary phase of the project, user requirements have been collected and uses cases, i.e., procedures, defined. Then, all the “elementary” operations needed to implement the use cases have been identified. Operations like screw/unscrew, lock/unlock, assemble/disassemble, connect/disconnect, thread/un-thread, etc. can be composed as elementary bricks to define a set of sequential steps/actions. A 3D CAD model of the machine tool has to be available. Each part of the model can be related to a specific action. Priority constraints can be also linked to actions in order to guarantee the exact procedure order. Each step is equivalent to a well defined state and the sequence of all the steps describes the whole state machine. In the vision of EASE-Formula$\text{R}^{3}$, new steps could be arranged at the headquarters, combined with existing ones and then sent to technicians. In this way, AR applications would be regarded no more as monolithic blocks, as they could be modified/enhanced on-the-fly in order to eventually manage unpredicted situations.

A 3-axis CNC machine may be considered as an example. The maintenance procedure to be supported by an AR application could be the replacement of the tool-holder. The standard AR-based procedure would be organized as illustrated in Fig. 3:

  • the CNC machine is recognized by the AR application; this event triggers the transition from the IDLE to the STEP 1 state;
  • during STEP 1, a set of virtual aids are presented to the technician; hints suggest to unlock the two tap drills that hold up the tool-holder;
  • when the tool-holder and the tap drills are recognized as a new object, the state machine moves to STEP 2;
  • in STEP 2, new virtual hints are presented to the technician, which suggest to unthread the tool-holder;
  • a new tool-holder will be then inserted and the tap drills will be locked again to the CNC machine; the procedure will continue until completion.
Figure 3
Fig. 3. Possible maintenance procedure for a CNC machine depicted as a state diagram: a) standard AR step sequence, and b) reconfigured step sequence.

Now, what could happen if the technician is not able to unlock one or both the tap drills? For instance, bolts might be oxidized and it might be impossible to apply the force needed without damaging the support guides.

A solution could be to remove the entire tool-holder block from the machine and then wrench the bolts. However, if this problem had been never experienced before, a traditional AR-based application would be of no help (since it would lack the required steps), whereas the EASE-Formula$\text{R}^{3}$ methodology would allow the headquarters to remotely reconfigure the procedure. In particular, three new steps would be added just before STEP 2, as illustrated in Fig. 3b.



A prototype architecture of the proposed framework is illustrated in Fig. 4. The framework has been developed by implementing an agile methodology. Specifically, four basic steps have been identified: user requirements collection and analysis, design, implementation and testing. The above approach has been chosen to reduce development times, keep machine tools builders continually involved in the process and deliver periodic (small) advances to be tested. Moreover, this work methodology is expected to be capable to reduce risks of failure and to guarantee that a prototype will be ready to use at project month 18 when demonstrators start.

Figure 4
Fig. 4. Prototype architecture of the proposed framework supporting AR-based reconfigurable maintenance procedures.

The client side is represented by a mobile AR application that is designed for being used by the onsite technician. The application has been built by using the Metaio SDK for Android. At present, it runs on a tablet device, but is scheduled to be migrated on a head-worn AR system for hands-free operations.

The server side consists of a Java-based desktop application, referred to as the (remote) control station, whose interface is reported in Fig. 5. On the top-left side of the interface, a graphics tool allows the operator sitting at the headquarters to define maintenance procedures, each one represented by a state machine. Edges represent state transitions, i.e., transitions between two consecutive procedure steps. A transition is described by the CAD model configuration that needs to be recognized in the scene by the AR system. The operator can assign to each edge an archive file containing the model associated with the particular transition. A preview of the model is also available (panel to the right of the state diagram). When, a new state is reached, the AR system is programmed to activate the associated virtual aids, or assets (at present, text hints, static/animated 3D models and videos). Assets can be linked to a specific state by means of the widget on the top-right side of the interface. The widget also allows the operator to configure the assets, e.g., by adjusting location, rotation and scale for 3D models to be displayed on screen.

Figure 5
Fig. 5. Interface of the remote control station used by the operator at the headquarters to create the AR maintenance procedures and reconfigure them as needed while providing assistance to onsite technicians: a) initial configuration of the state machine and b) reconfigured procedure.

The application workflow is as follows. When the AR application is launched on the client side device, it displays to the technician a list of procedures that are locally available (which were created offline using the above tool). The technician starts to follow the procedure by framing the scene with the device’s camera. When the AR system recognizes the CAD model for the current step, it starts tracking its pose and overlaps the associated 3D assets onto it. It also displays the text snippet on screen and uses text-to-speech to read it to the technician.

When the technician notifies to the application that the current step has been completed by pressing the right arrow icon, the AR system starts looking for the CAD model configuration associated with the next step. The choice to have an explicit notification of step completion was due to a specific user requirement about the possibility to move forward and backward in the procedure, as needed. Nonetheless, it is worth observing that this choice brought the additional advantage of limiting the number of false recognitions, which would result in the activation of wrong steps. The left arrow icon lets the technician go back to a previous step, whereas the reload icon forces the AR application to repeat recognition and assets presentation. A movie frame icon allows the technician to display a pre-recorded video file showing the execution of the particular step by an expert. Fig. 6 shows a screenshot of the interface of the AR application during experimental tests.

Figure 6
Fig. 6. Screenshot of the mobile AR application running on an Android tablet device captured during the execution of a maintenance procedure. Actions to be performed by the technician are described by the text snippet, which is augmented by static and animated 3D contents and text-to-speech audio.

In case of needs, the technician could use the designed system to ask the operator at the remote control station for assistance. The client application connects to the server side (e.g., on a Virtual Private Network, VPN) and sends information about the running procedure over a TCP/IP socket. The format used for transferring (and storing) the procedure is obtained by extending the XML schema defined by Metaio (to configure recognition, tracking and assets presentation) and adding elements for describing states and transitions.

When received by the server side, the state diagram appears in the interface of the desktop application and the current step is highlighted. By means of an audio communication and a text chat with the onsite technician, the operator at the headquarters may realize that the current procedure needs to be reconfigured in order to be properly completed. Hence, he or she could decide to adjust the procedure by removing existing nodes/edges, adding new ones and managing associated CAD models and assets accordingly. The operator could also decide to import other procedures available. When the reconfigured procedure is ready, the operator can send it to the client side, where it will be reloaded by the AR application. During remote assistance, the state machine on the client side application and the state diagram on the remote control station remain synchronized, in order to let the operator check the effectiveness of changes made and allow him or her to intervene again, if needed. A video of the overall workflow is available for download.2



At present, the AR procedures tailored to the machine tools developed by industrial partners involved in the EASE-Formula$\text{R}^{3}$ project (and related models and assets) are under development. Nonetheless, it was considered of extreme importance to start gathering experimental results about the system even during development, in order to fine-tune next implementation steps. Hence, a preliminary test campaign was initiated by working with the prototype architecture above and taking into account the maintenance of common use devices. Tests were carried out thanks to the participation of volunteers selected among Politecnico di Torino students.

A first session of tests was aimed at checking the functioning of the AR system alone (i.e., without the possibility to reconfigure the procedures), by comparing it to a traditional maintenance approach. The session involved 12 students (6 males and 6 females) enrolled in the BS degree in Visual Design, which were asked to execute the procedure for changing the hard disk of the netbook in Fig. 6 using the screwdrivers provided. Students had poor to none experience with this kind of operations.

Half of the students (3 males and 3 females) executed the test by following paper-based instructions. The other half executed the test using the AR procedure on the tablet device. For each student, the average time required to complete the test as well as possible errors made (wrong tool selected, or wrong step/action performed) were recorded. At the end of the test, students were asked to provide a qualitative feedback by filling in a questionnaire. Since the procedure was not particularly complex, it was unlikely to observe significantly different completion times. In fact, students spent, on average, 630.5s to complete the procedure with the AR-based system, which was only slightly less than the 671s required with paper-based instructions. However, it was observed that, with AR, standard deviation was much lower (102.6s) than with the traditional approach (172.5s). That is, AR smoothed the differences in experience, practical skills, etc. among the various students, making results more predictable. Furthermore, using the AR tool, the number of errors made reduced from 8 to just 3.

Based on promising results above, a second session of tests was carried out in order to measure the advantages possibly coming from exploiting the reconfigurability opportunities offered by the EASE-Formula$\text{R}^{3}$ prototype framework. The session involved 12 students (6 males and 6 females) from the MS in Ecodesign, which were asked to execute a two-phase procedure still on the above netbook. In the first phase, they had to change the memory bank of the netbook. Upon completion of the first phase, they were told that the netbook was still not working and were informed that a second phase had to be performed. The second phase consisted in replacing the hard disk by following the same procedure used in the first session. To gather comparable results, all the students executed the first phase with the AR system alone. Then, half of the students received the assistance required for completing the second phase through a voice/text-conference with the remote operator. The other half used the reconfigurable AR system, with the operator at the remote control station sending the new procedure to the client side.

Since for the first phase average completion time and standard deviation with the two approaches were roughly the same, the analysis was focused on the second phase. The number of errors made was almost comparable (3 with the AR system, 2 with voice/text conference). However, results about completion time were particularly interesting. In fact, by using the proposed reconfigurable AR system, students replaced the hard disk in 492s, on average (standard deviation 81s). For those who used the voice/text conference support only, time required was 20 percent higher (600s, on average, standard deviation 95s).

The encouraging evidences coming from the preliminary tests performed on the prototype system confirmed the great potential of using reconfigurable AR procedures as a complementary solution or an alternative to paper-based instructions and/or voice/text based tele-assistance.



Thanks to the reduced cost of technology and the increasing familiarity of average users with interactive computer and mobile applications, AR-based solutions for maintenance, repair and assembly are emerging as an interesting alternative to classical methodologies, e.g., based on paper-based instructions or electronic manuals. Nowadays, prototype systems are being studied and proposed by car manufacturers, furniture suppliers, etc.

This paper presents the past, the present and a possible future of AR-based maintenance. The evolution of AR technology is analyzed and new trends that could have an important social impact are presented. Open problems are also discussed, mainly focusing on technological issues that could limit a further spread of AR solutions. Finally, the framework that is being developed in the EASE-Formula$\text{R}^{3}$ project to address some of the open issues is introduced. Even though the project is primarily focused on the industrial context, it explores limitations that are shared by every AR application and proposes solutions that could be applied in different application domains. Specifically, it analyzes how the use of reconfigurable procedures that can be modified during AR-enabled tele-assistance sessions could improve the effectiveness of today’s maintenance processes.

Demonstrators will start at project month 18. The devised framework will be tested both on stone cutting machines manufactured by Wires Engineering and on milling machines produced by Fidia. In particular, for stone cutting machines four maintenance procedures will be supported in AR, concerning the substitution of the wheels and roller strips, of the sliding blocks of the needle carriage, of the wheel bearings and of the nitrogen piston. A further procedure will be implemented to support the maintenance of Fidia’s TMSC, a laser device for tool measurements. Experimental tests will be carried out by executing the AR procedures both on mobile devices as well as on special-purpose see-through glasses.

Thanks to the exploitation of AR-based tools, project partners estimate a reduction of at least 50 percent of national travels, and 40 percent of wordwide travels. Moreover, a reduction of repair times and costs up to 30 percent of on-site maintenance are expected. Future work will be aimed to complete the development of the framework and to test it on industrial machines by project partners, in order to gather both objective and subjective results under real operating conditions.


The authors would like to thank Dr. Antonio Gemma and Dr. Giuliano Pera for their contribution to the implementation of the prototype architecture.


EASE-R3 “Integrated Framework for a Cost-Effective and Ease of Repair, Renovation and Re-Use of Machine Tools Within Modern Factory” project (FP7-FoF.NMP.2013-8), Grant Agreement no. 608771.


No Data Available


Fabrizio Lamberti

Fabrizio Lamberti

Fabrizio Lamberti (M’02–SM’14) received the M.Sc. and Ph.D. degrees in computer engineering from the Politecnico di Torino, Turin, Italy, in 2000 and 2005, respectively, where he is currently an Associate Professor with the Dipartimento di Automatica e Informatica. He has authored a number of technical papers in international books, journals, and conferences in the areas of computer graphics, visualization, human-machine interaction, and mobile computing. He has served as a reviewer, a Program Committee Member, or an Organization Committee Member for several conferences. He is a member of the Editorial Advisory Board of international journals. He is a Senior Member of the IEEE Computer Society and a member of the IEEE Computer Society Technical Committee on Visualization and Graphics.

Federico Manuri

Federico Manuri

Federico Manuri received the B.Sc. and M.Sc. degrees in computer engineering from the Politecnico di Torino, Turin, Italy, in Italy, in 2008 and 2011, respectively, where he is currently pursuing the Ph.D. degree with the Dipartimento di Automatica e Informatica. His research interests include human-machine interaction, computer graphics, and augmented reality.

Andrea Sanna

Andrea Sanna

Andrea Sanna received the M.Sc. degree in electronic engineering and the Ph.D. degree in computer engineering from the Politecnico di Torino, Turin, Italy, in 1993 and 1997, respectively, where he is currently an Associate Professor with the Dipartimento di Automatica e Informatica. He has authored or co-authored several papers in the areas of computer graphics, virtual reality, distributed computing, and computational geometry. He is a Senior Member of the Association for Computing Machinery.

Gianluca Paravati

Gianluca Paravati

Gianluca Paravati received the B.Sc. and M.Sc. degrees in electronic engineering and the Ph.D. degree in computer engineering from the Politecnico di Torino, Turin, Italy, in 2005, 2007, and 2011, respectively, where he is currently an Assistant Professor with the Dipartimento di Automatica e Informatica. His research interests include real-time image processing, collaborative virtual environments, human-machine interaction, remote visualization, and distributed systems.

Pietro Pezzolla

Pietro Pezzolla

Pietro Pezzolla received the B.Sc. degree in electronic engineering from the Politecnico di Torino, Turin, Italy, in 2008, and the dual M.Sc. degree in electronic engineering from the Politecnico di Torino and Telecom ParisTech, Paris, France, in 2011. He is currently the Project Manager with Fidia S.p.A., Turin, high-speed milling machine tool builder. He is coordinating the European project EASER 3 integrated framework for a cost-effective and ease of repair, renovation, and reuse of machine tools within modern factory.

Paolo Montuschi

Paolo Montuschi

Paolo Montuschi (M’90–SM’07–F’14) is currently a Professor of Computer Engineering with the Politecnico di Torino, Turin, Italy, where he served as the Chair of Department from 2003 to 2011, and the Chair or a member of several boards, including the Board of Governors. He received the Ph.D. degree in computer engineering in 1989, and since 2000, he has been a Full Professor. He is serving as the Editor-in-Chief of the IEEE TRANSACTIONS ON COMPUTERS (2015–16), a Steering Committee Member and an Associate Editor of the IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTING, and an Advisory Board Member of Computing Now. He is also serving as a member of the IEEE Publication Services and Products Board. He served as the Chair of the Magazine Operations, the Electronic Products and Services, and the Digital Library Operations Committees, a Member-at-Large of the Computer Society’s Publications Board, and a member of the Board of Governors of the IEEE Computer Society. He served as a Guest Editor, an Associate Editor, and an Associate Editor-in-Chief of the IEEE TRANSACTIONS ON COMPUTERS from 2000 to 2004, 2009 to 2012, and 2013 to 2014, respectively, and the Co-Chair, a Program Committee Member, and a Steering Committee Member of several conferences. His current main research interests and scientific achievements are in computer arithmetic, computer architectures, computer graphics, electronic publications, semantics and education, and new frameworks for the dissemination of scientific knowledge. He is a Golden Core Member of the Computer Society and a Life Member of the International Academy of Sciences of Turin.

Cited By

No Data Available






Challenges, Opportunities, and Future Trends of Emerging Techniques for Augmented Reality-Based Maintenance Multimedia

This paper appears in:
No Data Available
Issue Date:
No Data Available
On page(s):
No Data Available
INSPEC Accession Number:
Digital Object Identifier:
Date of Current Version:
No Data Available
Date of Original Publication:
No Data Available

Text Size