Diffusion of the p-type dopant Mg in GaN and AlxGa1-xN which is accompanied by segregation and affected by transient effects in metal-organic vapor-phase epitaxy reactors is investigated. We have grown 110 nm thick Mg doped GaN and Al0.1Ga0.9N layers on top of undoped GaN and Al0.1Ga0.9N layers, respectively, in a temperature range between 925 °C and 1050 °C where we placed special emphasis on the lower temperature limit without diffusion to allow separation of Mg transients, diffusion, and segregation. Hereby, AlxGa1-xN layers enable monitoring of the resolution limit by secondary ion mass spectrometry analyses for the respective samples; therefore, thin AlxGa1-xN marker layers are incorporated in the thick GaN layers. We found an upper limit of 1.25 × 1019 cm-3 for diffusing Mg atoms in both sample types. Owing to the marked influence of Mg segregation in Al0.1Ga0.9N, diffusion is only seen by using a GaN cap on top of the Al0.1Ga0.9N layer sequence. Diffusion in Al0.1Ga0.9N is shown to be increased by about 25%-30% compared to GaN. Post growth annealing experiments under conditions equivalent to those used for growth of the Mg doped samples showed negligible diffusion. Comparing the results to well established findings on other doped III-V compounds, diffusion is explained by an interstitial-substitutional mechanism with a diffusion coefficient, which is concentration dependent. Analysis of the temperature dependent diffusivity revealed an activation energy of 5.0 eV for GaN:Mg and 5.2 eV for Al0.1Ga0.9N:Mg.