Most studies on adaptive coordination of multi-robot systems assume exact knowledge of system kinematics and deal only with dynamic uncertainties. However, many industrial applications involve tasks in which a multi-robot system interacts with geometrically unknown environments. In this paper, we consider a multi-robot system grasping a rigid object in contact with a geometrically unknown surface. The proposed adaptive hybrid force-motion controller guarantees asymptotic tracking of desired motion and force trajectories while ensuring exact identification of constraint Jacobian matrix without persistency of excitation condition. The control signal is smooth and does not depend on contact force derivative. The proposed adaptive controller is robustified against environmental friction and nonparametric uncertainty in environment geometry. Simulation examples are presented to illustrate the results.