Variable angle spectroscopic ellipsometry (VASE) was used to nondestructively characterize a silicon nitride (Si3N4)/silicon oxynitride (SiOxNy)/silicon dioxide (SiO2) thin film multilayer structure, in which Si3N4 and SiOxNy layers were deposited by plasma enhanced chemical vapor deposition and low‐pressure chemical vapor deposition, respectively. The measured VASE spectra (3500–8200 Å) were analyzed with an appropriate multilayer fitting model, in which the SiOxNy layer was modeled, in the Bruggeman effective medium approximation (EMA), to be a physical mixture of two distinct phases, SiO2 and Si3N4. Remarkably good agreement between the measured spectra and the model calculations was obtained, indicating that the EMA and the fitting model were appropriate for the data analysis. As a result, the three layer thicknesses, compositions, and the optical constants of SiOxNy layer at several spots across the sample were determined. The thickness values obtained by VASE were in very good agreement with those measured by cross‐sectional transmission electron microscopy.