The paper gives a BP neural network (BPNN) prediction model of the ambient air quality based on rough set theory. We make first the reduction of monitoring data of the pollution sources using the theory of rough set, extract the tidy rules. Then the topological structure of the multilayer BPNN and the nerve cells of the connotative layer are defined with these rules. After that the connected weight values of corresponding nodes of the BPNN are ascertained. Using BP arithmetic, the prediction model is trained with the monitoring data of the pollution sources and air monitor stations for gaining the various parameters of it. Finally, the model after training is used to predict the urban air quality with certain meteorological parameters. The result of the prediction model was proved that it is more accurate than the common BPNN.