We propose a method for compressing color map images by context tree modeling and arithmetic coding. We consider multicomponent map images with semantic layer separation and images that are divided into binary layers by color separation. The key issue in the compression method is the utilization of interlayer correlations, and to solve the optimal ordering of the layers. The interlayer dependencies are acquired by optimizing the context tree for every pair of image layers. The resulting cost matrix of the interlayer dependencies is considered as a directed spanning tree problem and solved by an algorithm based on the Edmond's algorithm for optimum branching and by the optimal selection and removal of the background color. The proposed method gives results 50% better than JBIG and 25% better than a single-layer context tree modeling.