In this paper, a vision-based algorithm for estimating tip interaction forces on a deflected atomic orce microscope (AFM) cantilever is described. Specifically, we propose that the algorithm can estimate forces acting on an AFM cantilever being used as a nanomanipulator inside a scanning electron microscope (SEM). The vision-based force sensor can provide force feedback in real time, a feature absent in many SEMs. A methodology based on cantilever slope detection is used to estimate the forces acting on the cantilever tip. The technique was tested on a scaled model of the nanoscale AFM cantilever and verified using theoretical estimates as well as direct strain measurements. Artificial SEM noise was introduced in the macroscale images to characterize our sensor under varying levels of noise and other SEM effects. Prior knowledge about the cantilever is not required, and the algorithm runs independent of human input. The method is shown to be effective under varying noise levels, and demonstrates improving performance as magnification levels are decreased. Therefore, we conclude that the vision-based force sensing algorithm is best suited for continuous operation of the SEM, fast scanning rates, and large fields-of-view associated with low magnification levels.