We report the preparation of microcrystalline ZnO thin films on sapphire substrates using a simple method based on the thermal evaporation of metallic Zn in vacuum with further annealing process. The aim of annealing in the oxygen atmosphere in the range of 800–850 °C was to obtain the high quality ZnO films. The surface morphology was studied by scanning electron microscopy and atomic force microscopy. The polycrystalline films with ZnO microrods at different stages of their growth were investigated. Second and third harmonic generation measurements were performed by means of the rotational Maker fringe technique using Nd:YAG laser at 1064 nm in picosecond regime. The obtained values of second and third order nonlinear susceptibilities were found to be high enough for the potential applications of the investigated materials in the optical switching devices based on refractive index changes.