In this article several kinetic effects are proposed that induce compositional instabilities in thick InGaN heteroepitaxial layers on GaN templates grown by metalorganic chemical vapor deposition. It was found that by reducing the growth temperature, or increasing the growth rate, or introducing Mg doping, the epitaxial layer changes from a pseudomorphic InGaN with a low indium mole fraction to a relaxed InGaN with a high indium mole fraction. In certain circumstances, both phases can be present in a single layer. The composition and strain inhomogeneity was correlated to the surface morphology and crystalline quality, governed by the growth conditions. It is believed that the compositional instability in InGaN originates from the coupled effects of compressive strain and surface morphology. A smooth surface allows for the growth of pseudomorphic low-indium InGaN, whereas a rough surface promotes the formation of a relaxed high-indium InGaN layer.