We have studied the nanoscale changes in morphology and optical properties during annealing for bulk-heterojunction poly(3-hexylthiophene) (P3HT) and [6,6]-phenyl C61-butyric acid methyl ester (PCBM) composite film. Thermal atomic force microscopy was used to monitor the morphology evolution of the film in situ quantitatively, which showed a migration and aggregation of PCBM with increasing temperature. Scanning near-field microscopy was used to investigate the quantitative changes in absorption behavior of the film in nanoscale with increasing annealing time at 140 °C, which indicated that the extent of absorption of the film was increased with increasing annealing time. However, a large PCBM aggregate (1 μm) was formed after the film annealed at 140 °C for 1 h. The aggregate interrupted the bicontinous morphology of the film and further affected the absorption behavior in nanoscale. Furthermore, the refractive index and extinction coefficient of the films increased after annealed 30 min at 140 °C, but decreased after an extended annealing for 60 min. The increase in optical properties indicated the film achieving a highly ordered structure upon heating. The results suggested that an optimized annealing process was at 140 °C for 30 min. Finally, devices with different annealing times at 140 °C were fabricated and evaluated. The highest charge mobility and power conversion efficiency of the device were fabricated as suggested annealing conditions. The nanoscale monitoring of the P3HT/PCBM film has been found to be very useful to determine the optimized annealing conditions for high efficiency photovoltaic device.