In force control applications of robots, it is difficult to obtain an exact model of a constraint surface. In presence of the constraint uncertainty, the robot needs to adapt to the uncertainty in external parameters due to the environment, in addition to the uncertainties in internal parameters of the robot kinematics and dynamics. In this paper, a visually servoed adaptive controller is proposed for motion and force tracking with uncertainties in the constraint surface, kinematics, dynamics, and camera model. We shall show that the robot can track the desired trajectories with the uncertain internal and external parameters updated online. This gives the robot a high degree of flexibility in dealing with changes and uncertainties in its model and the environment.