Detection of task-specific moving objects from a sequence of images obtained by a camera attached to a moving robot is a complex task. This is mainly due to background motion in the video. This paper proposes a probabilistic object-based motion attention model for this purpose. This model is composed of four components: pre-attentive object-based segmentation, bottom-up motion attention, object-based top-down biasing, and contour based object representation. The object-based attentional competition is implemented by combination of bottom-up saliency and top-down bias maps. The probabilistic distribution of attention is obtained by using Bayesian inference so as to allow uncertainties to be present. The proposed method can directly stand out moving objects of interest, thus the necessity of background motion estimation is eliminated. Experimental results in natural scenes have shown to validate this method even in case of occlusion.