Key-frame Extraction With Semantic Graphs in Assembly Processes

In this letter, we present a novel method for key-frame extraction in assembly processes, with the support of visual sensors. The method is based on the construction of a sequence of weighted graphs containing the semantics of the underlying scene, encoded as primitive temporal and spatial relations between the involved objects. The objects are modeled as three-dimensional ellipsoids, with the formulation of the minimum volume covering ellipsoid problem as a convex optimization program, which is solved by an efficient algorithm. The new models are used in the sequence of actions–relations to increase the accuracy of the key-frame extraction, by extending the states encoded in the graphs. We present and analyze the method, followed by preliminary experimental results, and provide significant evidence on its potential.