In this paper the adaptive feature selection model (AFSM) which is based on two layers, adaptive and multi-classes JM distance is studied. During the process of crop identification using remote sensing (RS) image classification, it is the effective way to improve the classification accuracy that the feature is proper treated. Firstly, with MODIS data as examples, the extracted spectral characteristics are analyzed using statistical method and dynamic changes of temporal series of indices including NDVI, EVI, MSAVI and NDWI are studied. Secondly, the rice is chosen as the experimental object and the theory of multi-objective planning of operation research is introduced. Then AFSM is developed. In order to improve recognition accuracy, the objects are divided into two groups: the upper-level objection and the lower-level objection and target factors are defined, and the JM distance among two classes in the upper-level objection is adjusted by the degree of difficulty to identify the classes. Finally, the genetic algorithm is adopted to improve the search speed and accuracy of obtaining the optimal feature selection. This model is not only feasible, helpful to improve the accuracy of crop identification by the proof of experiments on rice identification in Songyuan city of Jilin province, Northeast China, but also applied to the primary crop investigation using RS at a large scale.