Despite the recent progress in physically interactive and surgical robotics, the active deformation of compliant objects remains an open problem. The main obstacle to its implementation comes from the difficulty to identify or estimate the object's deformation model. In this paper, we propose a novel vision-based deformation controller for robot manipulators interacting with unknown elastic objects. We derive a new dynamic-state feedback velocity control law using the passivity-based framework. Our method exploits visual feedback to estimate the deformation Jacobian matrix in real time, avoiding any model identification steps. We prove that even in the presence of inexact estimations, the closed-loop dynamical system ensures input-to-state stability (i.e., full dissipativity) with respect to external disturbances. An experimental study with several deformation tasks is presented to validate the theory.