Research Papers

Cloud-Based Augmented Reality Remote Maintenance Through Shop-Floor Monitoring: A Product-Service System Approach

[+] Author and Article Information
Dimitris Mourtzis

Laboratory for Manufacturing Systems
and Automation,
University of Patras,
Patras 26500, Greece
e-mail: mourtzis@lms.mech.upatras.gr

Aikaterini Vlachou

Laboratory for Manufacturing Systems
and Automation,
University of Patras,
Patras 26500, Greece
e-mail: vlachou@lms.mech.upatras.gr

Vasilios Zogopoulos

Laboratory for Manufacturing Systems
and Automation,
University of Patras,
Patras 26500, Greece
e-mail: zogopoulos@lms.mech.upatras.gr

1Corresponding author.

Manuscript received October 24, 2016; final manuscript received January 10, 2017; published online January 30, 2017. Editor: Y. Lawrence Yao.

J. Manuf. Sci. Eng 139(6), 061011 (Jan 30, 2017) (11 pages) Paper No: MANU-16-1561; doi: 10.1115/1.4035721 History: Received October 24, 2016; Revised January 10, 2017

Maintenance and its cost continue, over the years, drawing the attention of production management, since the unplanned failures decrease the reliability of the system and the return of investments. Maintenance services of manufactured products are among the most common services in the industry; they account for more than half of the total costs and influence the environmental impact of the product. In order for manufacturers to increase their productivity, by performing accurate and quick maintenance, advanced monitoring systems should be considered in order to easily detect machine tool failures before they occur. Toward that end, a cloud-based platform for condition-based preventive maintenance, supported by a shop-floor monitoring service and an augmented reality (AR) application, is proposed as a product-service system (CARM2-PSS). The proposed AR maintenance service consists of algorithms of automated generation of assembly sequences, part movement scripts, and improved interface that aim to maximize existing knowledge usage while creating vivid AR service instructions. Moreover, the proposed monitoring system is supported by a wireless sensor network (WSN), and is deployed on a Cloud environment together with the AR tool. The monitoring system monitors the status of the machine tools, calculates their remaining operating time between failures (ROTBF), and identifies the available windows of the machine tools in order to perform the AR remote maintenance. In order to validate the proposed methodology and calculate its impact, it is applied in a real-life case study of a white-goods industry.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.


Chryssolouris, G. , 2013, Manufacturing Systems: Theory and Practice, Springer Science & Business Media, New York.
Efthymiou, K. , Papakostas, N. , Mourtzis, D. , and Chryssolouris, G. , 2012, “ On a Predictive Maintenance Platform for Production Systems,” Proc. CIRP, 3, pp. 221–226. [CrossRef]
Mori, M. , Fujishima, M. , Komatsu, M. , Zhao, B. , and Liu, Y. , 2008, “ Development of Remote Monitoring and Maintenance System for Machine Tools,” CIRP Ann.-Manuf. Technol., 57(1), pp. 433–436. [CrossRef]
Nee, A. Y. C. , Ong, S. K. , Chryssolouris, G. , and Mourtzis, D. , 2012, “ Augmented Reality Applications in Design and Manufacturing,” CIRP Ann.-Manuf. Technol., 61(2), pp. 657–679. [CrossRef]
Mourtzis, D. , and Doukas, M. , 2014, “ The Evolution of Manufacturing Systems: From Craftsmanship to the Era of Customisation,” Handbook of Research on Design and Management of Lean Production Systems, IGI Global, Hershey, PA.
Schmidt, B. , Galar, D. , and Wang, L. , 2016, “ Big Data in Maintenance Decision Support Systems: Aggregation of Disparate Data Types,” Euro Maintenance Conference Proceedings, Athens, Greece, May 30–June 1, pp. 503–512.
Xu, X. , 2012, “ From Cloud Computing to Cloud Manufacturing,” Rob. Comput.-Integr. Manuf., 28(1), pp. 75–86. [CrossRef]
Adamson, G. , Wang, L. , and Holm, M. , 2015, “ Cloud Manufacturing–A Critical Review of Recent Development and Future Trends,” Int. J. Comput. Integr. Manuf., pp. 1–34.
Mourtzis, D. , and Vlachou, E. , 2016, “ Cloud-Based Cyber-Physical Systems and Quality of Services,” TQM J., 28(5), pp. 704–733. [CrossRef]
Takata, S. , Kirnura, F. , Van Houten, F. J. A. M. , Westkamper, E. , Shpitalni, M. , Ceglarek, D. , and Lee, J. , 2004, “ Maintenance: Changing Role in Life Cycle Management,” CIRP Ann.-Manuf. Technol., 53(2), pp. 643–655. [CrossRef]
Dhilon, B. S. , 2006, Maintenability, Maintenance, and Reliability for Engineers, Taylor and Francis, New York.
Venkataraman, V. , 2007, Maintenance Engineering and Management, PHI Learning Pvt. Ltd., New Delhi, India.
Sutherland, I. E. , 1968, “ A Head-Mounted Three Dimensional Display,” Fall Joint Computer Conference—Part: I, San Francisco, CA, Dec. 9–11, pp. 757–764.
Feiner, S. , Macintyre, B. , and Seligmann, D. , 1993, “ Knowledge-Based Augmented Reality,” Commun. ACM, 36(7), pp. 53–62. [CrossRef]
Schwald, B. , and De Laval, B. , 2003, “ An Augmented Reality System for Training and Assistance to Maintenance in the Industrial Context,” J. WSCG, 11(1–3), pp. 425–432.
Henderson, S. J. , and Feiner, S. , 2007, Augmented Reality for Maintenance and Repair (Armar), Department of Computer Science, Columbia University, New York.
Didier, J.-Y. , Roussel, D. , Mallem, M. , Otmane, S. , Naudet, S. , Pham, Q. C. , Bourgeois, S. , Mégard, C. , Leroux, C. , and Hocquard, A. , 2005, “ AMRA: Augmented Reality Assistance for Train Maintenance Tasks,” In Workshop Industrial Augmented Reality,” 4th ACM/IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Vienna, Austria, Oct. 5–8.
Ong, S. K. , Yuan, M. L. , and Nee, A. Y. C. , 2008, “ Augmented Reality Applications in Manufacturing: A Survey,” Int. J. Prod. Res., 46(10), pp. 2707–2742. [CrossRef]
Platonov, J. , Heibel, H. , Meier, P. , and Grollmann, B. , 2006, “ A Mobile Markerless AR System for Maintenance and Repair,” 5th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR), Santa Barbara, CA, Oct. 22–25, pp. 105–108.
Bottecchia, S. , Jean-Marc, C. , and Jean-Pierre, J. , 2010, “ TAC: Augmented Reality System for Collaborative Tele-Assistance in the Field of Maintenance Through Internet,” 1st Augmented Human International Conference, Megève, France, Apr. 2–03, p. 14.
Leila, A. , Tecchia, F. , and Huang, W. , 2011, “ Remote Tele-Assistance System for Maintenance Operators in Mines,” 11th Underground Coal Operators' Conference, Wollongong, NSW, Australia, pp. 171–177.
Gao, R. , Wang, L. , Teti, R. , Dornfeld, D. , Kumara, S. , Mori, M. , and Helu, M. , 2015, “ Cloud-Enabled Prognosis for Manufacturing,” CIRP Ann.-Manuf. Technol., 64(2), pp. 749–772. [CrossRef]
Koochaki, J. , Bokhorst, O. C. , Wortmann, H. , and Klingenberg, W. , 2012, “ Condition Based Maintenance in the Context of Opportunistic Maintenance,” Int. J. Prod. Res., 50(23), pp. 6918–6929. [CrossRef]
Knapp, G. M. , and Wang, H.-P. , 1996, “ Automated Tactical Maintenance Planning Based on Machine Monitoring,” Int. J. Prod. Res., 34(3), pp. 753–765. [CrossRef]
Colledani, M. , Tullio, T. , Anat, F. , Benoit, I. , Lanza, G. , Schmitt, R. , and Vancza, J. , 2014, “ Design and Management of Manufacturing Systems for Production Quality,” CIRP Ann.-Manuf. Technol., 63(2), pp. 773–796. [CrossRef]
Teti, R. , Jemielniak, K. , O'Donnell, G. , and Dornfeld, D. , 2010, “ Advanced Monitoring of Machining Operations,” CIRP Ann.-Manuf. Technol., 59(2), pp. 717–739. [CrossRef]
Mourtzis, D. , Vlachou, E. , Xanthopoulos, N. , Givehchi, M. , and Wang, L. , 2016, “ Cloud-Based Adaptive Process Planning Considering Availability and Capabilities of Machine Tools,” J. Manuf. Syst., 39, pp. 1–8. [CrossRef]
Mehnen, J. , Tinsley, L. , and Roy, R. , 2014, “ Automated In-Service Damage Identification,” CIRP Ann.-Manuf. Technol., 63(1), pp. 33–36. [CrossRef]
Li, X. , Wang, L. , and Cai, N. , 2004, “ Machine Vision Based Surface Finish Inspection for Cutting Tool Replacement in Production,” Int. J. Prod. Res., 42(11), pp. 2279–2287. [CrossRef]
Tapoglou, N. , Mehnen, J. , Vlachou, E. , Doukas, M. , Milas, N. , and Mourtzis, D. , 2015, “ Cloud-Based Platform for Optimal Machining Parameter Selection Based on Function Blocks and Real-Time Monitoring,” ASME J. Manuf. Sci. Eng., 137(4), p. 040909. [CrossRef]
Mourtzis, D. , Doukas, M. , Vlachou, E. , and Xanthopoulos, N. , 2014, “ Machine Availability Monitoring for Adaptive Holistic Scheduling: A Conceptual Framework for Mass Customization,” Proc. CIRP, 25, pp. 406–413. [CrossRef]
Mourtzis, D. , Fotia, S. , and Doukas, M. , 2015, “ Performance Indicators for the Evaluation of Product-Service Systems Design: A Review,” IFIP International Conference on Advances in Production Management Systems, Springer International Publishing, Tokyo, Japan, pp. 592–601.
Schmidt, B. , and Wang, L. , 2016, “ Cloud-Enhanced Predictive Maintenance,” J. Adv. Manuf. Technol., pp. 1–9.
Wang, L. , 2013, “ Machine Availability Monitoring and Machining Process Planning Towards Cloud Manufacturing,” CIRP J. Manuf. Sci. Technol., 6(4), pp. 263–273. [CrossRef]
Edrington, B. , Bingyan, Z. , Hansel, A. , Mori, M. , and Fujishima, M. , 2014, “ Machine Monitoring System Based on MTConnect Technology,” Proc. CIRP, 22, pp. 92–97. [CrossRef]
Zhang, W. , Halang, W. , and Diedrich, C. , 2003, “ An Agent-Based Platform for Service Integration in E-Maintenance,” IEEE International Conference on Industrial Technology, Maribor, Slovenia, Dec. 10–12, Vol. 1, pp. 426–433.
Mourtzis, D. , Schoinochoritis, B. , and Vlachou, E. , 2015, “ A New Era of Web Collaboration: Cloud Computing and Its Applications in Manufacturing,” 8th IWC Total Quality Management Advanced and Intelligent Approaches, Belgrade, Serbia, June 1–5, pp. 11–23.
Buckholtz, B. , Ragai, I. , and Wang, L. , 2016, “ Remote Equipment Security in Cloud Manufacturing Systems,” Int. J. Manuf. Res., 11(2), pp. 126–143. [CrossRef]
Mourtzis, D. , Fotia, S. , and Vlachou, E. , 2016, “ PSS Design Evaluation Via KPIs and Lean Design Assistance Supported by the Context Sensitivity Tools,” Proc. CIRP, 56, pp. 496–501.
Buyya, R. , Yeo, C. S. , Venugopal, S. , Broberg, J. , and Brandic, I. , 2009, “ Cloud Computing and Emerging IT Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility,” Future Gener. Comput. Syst., 25(6), pp. 599–616. [CrossRef]
Mourtzis, D. , Vlachou, E. , Milas, N. , and Xanthopoulos, N. , 2016, “ A Cloud-Based Approach for Maintenance of Machine Tools and Equipment Based on Shop-Floor Monitoring,” Proc. CIRP, 41, pp. 655–660. [CrossRef]
Mourtzis, D. , Vlachou, E. , Doukas, M. , Kanakis, N. , Xanthopoulos, N. , and Koutoupes, A. , 2015, “ Cloud-Based Adaptive Shop-Floor Scheduling Considering Machine Tool Availability,” ASME Paper No. IMECE2015-53025.
Pintzos, G. , Triantafyllou, C. , Papakostas, N. , Mourtzis, D. , and Chryssolouris, G. , 2016, “ Assembly Precedence Diagram Generation Through Assembly Tiers Determination,” Int. J. Comput. Integr. Manuf., 29(10), pp. 1045–1057. [CrossRef]
Baines, T. , Lightfoot, H. W. , Evans, S. , Neely, A. , Greenough, R. , Peppard, J. , and Wilson, H. , 2007, “ State-of-the-Art in Product-Service Systems,” Proc. Inst. Mech. Eng., Part B, 221(10), pp. 1543–1552. [CrossRef]
Azarenko, A. , Roy, R. , Shehab, E. , and Tiwari, A. , 2009, “ Technical Product-Service Systems: Some Implications for the Machine Tool Industry,” J. Manuf. Technol. Manage., 20(5), pp. 700–722. [CrossRef]
Vuzix, 2012, “ Vuzix STAR 1200XL: Product Specifications,” Vuzix, Rochester, NY.
LG, 2012, “ Nexus 4: Product Specifications,” LG, Amstelveen, The Netherlands.
Unity, 2016, Unity Technologies, San Francisco, CA, accessed Dec. 10, 2016, https://unity3d.com/
Milgram, P. , Takemura, P. , Utsumi, A. , and Kishino, F. , 1995, “ Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum,” Photonics for Industrial Applications, International Society for Optics and Photonics, Bellingham, WA, pp. 282–292.
Siltanen, S. , 2012, “ Theory and Applications of Marker-Based Augmented Reality,” VTT Science, Espoo, Finland.
Vuforia, 2015, PTC Inc., Boston, MA, accessed Dec. 9, 2016, https://www.vuforia.com/


Grahic Jump Location
Fig. 1

The CARM2-PSS approach

Grahic Jump Location
Fig. 2

Shop-floor monitoring service

Grahic Jump Location
Fig. 3

The AR remote maintenance service and the smart assembly/disassembly precedence algorithm

Grahic Jump Location
Fig. 4

The proposed product-oriented PSS model

Grahic Jump Location
Fig. 5

Augmented reality application

Grahic Jump Location
Fig. 6

Current maintenance approach versus CARM2-PSS platform for condition-based preventive maintenance

Grahic Jump Location
Fig. 7

Application of the CARM2-PSS platform

Grahic Jump Location
Fig. 8

Maintenance time (MT), maintenance cost (MC), and usability (US) of proposed versus current maintenance approach




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In