We present a new image segmentation algorithm based on a tree-structured binary MRF model. The image is recursively segmented in smaller and smaller regions until a stopping condition, local to each region, is met. Each elementary binary segmentation is obtained as the solution of a MAP estimation problem, with the region prior modeled as an MRF. Since only binary fields are used, and thanks to the tree structure, the algorithm is quite fast, and allows one to address the cluster validation problem in a seamless way. In addition, all field parameters are estimated locally, allowing for some spatial adaptivity. To improve segmentation accuracy, a split-and-merge procedure is also developed and a spatially adaptive MRF model is used. Numerical experiments on multispectral images show that the proposed algorithm is much faster than a similar reference algorithm based on "flat" MRF models, and its performance, in terms of segmentation accuracy and map smoothness, is comparable or even superior.