Grasping the Performance: Facilitating Replicable Performance Measures via Benchmarking and Standardized Methodologies

It is comparatively easy to make computers exhibit adult-level performance on intelligence tests or playing checkers, and difficult to give them the skills of a one-year-old when it comes to perception and dexterity. More than 15 years after it was first stated, Moravec's paradox still holds true today. Fueled by vigorous research in machine learning, the gap has consistently narrowed on the perception side. However, most of the fine manual motor skills displayed by a toddler are, to date, far beyond what robots can do. It is true that many valuable tasks involving physical interaction with objects can be solved by contemporary robots as indicated by a thriving industrial robotics sector. However, in the future, robots are expected to work side by side with humans in unstructured environments, and the ability to reliably grasp and manipulate objects used in everyday activities will be an unavoidable requirement. Today's robots are far from being ready for this challenge.