0
Research Papers

Computationally Efficient Optimal Video Comparison for Machine Monitoring and Process Control

[+] Author and Article Information
Brian W. Anthony

Mem. ASME
Laboratory for Manufacturing and Productivity,
Department of Mechanical Engineering,
Massachusetts Institute of Technology,
Cambridge, MA 02139
e-mail: banthony@mit.edu

Fitriani Chua

Center for Design Optimization,
Massachusetts Institute of Technology,
Cambridge, MA 02139
e-mail: fitriani@mit.edu

1Corresponding author.

Manuscript received January 31, 2017; final manuscript received June 16, 2017; published online August 24, 2017. Assoc. Editor: Robert Gao.

J. Manuf. Sci. Eng 139(10), 101007 (Aug 24, 2017) (11 pages) Paper No: MANU-17-1060; doi: 10.1115/1.4037234 History: Received January 31, 2017; Revised June 16, 2017

Real-time algorithms are needed to compare and analyze digital videos of machines and processes. New video analysis techniques, for computationally efficient dimensionality-reduction, for determination of accurate motion-information, and for fast video comparison, will enable new approaches to system monitoring and control. We define the video alignment path (VAP) as the sequence of local time-and-space transformations required to optimally register two video clips. We develop an algorithm, dynamic time and space warping (DTSW), which calculates the VAP. Measures of video similarity, and therefore system similarity, are estimated based on properties of the VAP. These measures of similarity are then monitored over time and used for decision-making and process control. We describe the performance, structure, and computational complexity of a DTSW implementation, which is parallelizable and which can achieve the processing rates necessary for many video-based industrial monitoring applications. We describe two case studies of unsupervised monitoring for mechanical wear and for fault detection. Results suggest opportunities for boarder applications of video-based instrumentation for real-time feedback control, wear and defect detection, or statistical process control.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.

References

Figures

Grahic Jump Location
Fig. 1

Video alignment path. The VAP is the time-and-space path that one video segment, Q, follows through another, C.

Grahic Jump Location
Fig. 2

(a) Implementation—eigenframes as filters. Instead of filtering the test sequence with every frame of the template sequence, we filter it with the truncated set of basis frames. The outputs from these basis filters are then linearly combined using the projection coefficients to approximately determine the terms in the elemental distance hypervolume. (b) Parallel implementation of DTSW.

Grahic Jump Location
Fig. 3

Heart valve example. C is a video of a heart valve cycle toward the end of the lifetime test—an “old” valve. Q is a video of the heart valve at the very beginning of the lifetime test—a “new” valve. The bottom two strips show the videos— Cw* and Qw—output from the DTSW algorithm.

Grahic Jump Location
Fig. 4

Eigenframes and reconstruction coefficients for heart valve example

Grahic Jump Location
Fig. 5

W* (the VAP) of synthetic heart valve at beginning and end of lifetime test

Grahic Jump Location
Fig. 6

Local (d(wk)) and cumulative (d0*(wk)) distances along W* (VAP) for synthetic heart valve at beginning and end of lifetime test

Grahic Jump Location
Fig. 7

Relevant spatial planes of the local distance hypervolume, DL, with optimal warp path. DTSW results for an old heart valve video, C, and a new heart valve video, Q. The solid black line is the temporal projection of W* onto each spatial plane; the dots indicate that the spatial component of the W* is in the indicated spatial plane.

Grahic Jump Location
Fig. 8

Temporal and spatial error of synthetic heart valve at beginning and end of lifetime test

Grahic Jump Location
Fig. 9

Heart valve event summary time error during lifetime test. We have eliminated a large section of time between the newer and older valves.

Grahic Jump Location
Fig. 10

Diaper packaging example. Test, C, and template, Q, input video sequences are shown in the top of figure. The bottom two strips show the videos— Cw* and Qw—output from the DTSW algorithm.

Grahic Jump Location
Fig. 11

Eigenframes and reconstruction coefficients for diaper packaging monitoring example

Grahic Jump Location
Fig. 12

Diaper wrapping event summary statistics: d0*/J —normalized DTSW distance, SIL2 time error, SXYL2 position error, prior to, including, and after the first glitch at Event n are plotted. In order to autonomously monitor and detect faults, we would first set a threshold during normal operation and then detect when that threshold is exceeded. We see that it would be easy to select thresholds that detect a problem starting at Event n and through Event n + 4. The error bars show the errors as a function of preserved variance using an Eigenframe approximation of Q. Here, we show the induced variation for 98%, 81%, and 62% preserved variance. This corresponds to 18, 5, and 2 eigenframes.

Grahic Jump Location
Fig. 13

Local structure of D0 for the heart valve example. The partial derivative of D0 along the j temporal direction, ∂D0/∂j, is small in the range of frames 17–19.

Grahic Jump Location
Fig. 14

Local structure of D0 for the heart valve example. The partial derivative of D0 along the x and y spatial directions, ∂D0/∂x and ∂D0/∂y, are small at the temporal beginning and end of the sequence.

Grahic Jump Location
Fig. 15

Heart valve event-summary run chart during lifetime test. The wider error bars indicate decreased preserved variance. Here, we show the induced variation for 90%, 80%, and 60% preserved variance. This corresponds to eight, four, and two eigenframes. We have eliminated a large section of time between the new and old, cycle count ≥ N, valves. For this example, the change in temporal component is most critical, but for completeness we show the position error and DTSW distance changes.

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In