Compositional, structural, and optical properties of molecular-beam epitaxy grown AlxGa1-xN films were characterized by transmission electron microscopy (TEM), x-ray diffraction, and cathodoluminescence spectroscopy. Spontaneous modulation, phase separation, and band gap reductions were observed to vary systematically with AlN mole fraction across the full alloy series. At low AlN mole fraction (x≤0.5), AlGaN epilayers display pronounced phase separation. With increasing AlN mole fraction, phase separation is strongly suppressed by the formation of spontaneous modulation which high spatial resolution TEM techniques unambiguously determine to be atomic-scale compositional superlattice. The formation of the spontaneous superlattice is considered responsible for the pronounced reductions in band gaps and emission energies, exceeding several hundred meV for the Al-rich AlGaN, which has been confirmed by band structure calculations.