A theory for treating uncertainty in localizing 3D objects using local, area — based stereo algorithms is presented. In addition, the resultant implications for practical applications are discussed. First, we propose a phase — based stereo algorithm as a representative of this kind of algorithms. The concept of a receptive field is introduced to describe a region on the target where stimuli with fixed disparity but variable position in the receptive field yield the same disparity response. This positional uncertainty on the targets is transformed into a localization uncertainty existing in the world. Strategies for optimally localizing 3D structure are developed. These considerations also lead to an upper bound for the vergence angle and a fusion strategy for multiple views when employing an active stereo camera system. Finally we show the validity of our approach by experimental results.