Terahertz time-domain spectroscopy has been employed to measure the optical constants of ytterbium-doped yttrium aluminum garnet (YbxY1-x)3Al5O12 (Yb:YAG) crystals for nominal x values of 0.0, 0.1, 0.2, 0.5, 0.8, and 1.0 in the frequency range from 0.2 to 1.8 THz. The real refractive indices for Yb:YAG crystals increase linearly with the concentration of Yb3+. The experimental results can be fitted by the Sellmeier equation. The results imply that the phonon modes of the Yb:YAG crystals shift to lower frequencies with substitution of Yb for Y. The extinction coefficients of the Yb:YAG crystals are smaller than 0.05.