We report on the use of in-situ SiNx nanomask for defect reduction in nonpolar a-plane GaN films, grown by metal-organic chemical vapor deposition. High-resolution x-ray diffraction analysis revealed that there was a monotonic reduction in the full width at half maximum, both on-axis and off-axis, with the increase in the SiNx thickness. Atomic force microscopy images revealed a significant decrease in the root-mean-square roughness and the density of submicron pits. Cross-section and plan-view transmission electron microscopy on the samples showed that the stacking fault density decreased from 8×105 to 3×105 cm-1 and threading dislocation density decreased from 8×1010 to 9×109 cm-2. Room temperature photoluminescence measurement revealed that the band-edge emission intensity increased with the insertion of the SiNx layer, which suggests reduction in the nonradiative recombination centers.