The performance of CdZnTe room-temperature X-ray and gamma-ray detectors is determined by material and device defects that govern carrier transport trough the device. In this contribution, we review common bulk, interface, and surface defects and their effects on charge transport, charge transport uniformity, and device performance. We note that pure CdZnTe grown under Te-rich conditions has an excess of Cd-vacancies and other Te-related native defects and must be electrically compensated in order to obtain high resistivity material. Through the critical analysis of the various compensation schemes it is shown that deep level defects must be introduced with donor doping elements in order to achieve a practical compensation technique. The role of carrier trapping and limitations on detector performance with increasing crystal size are discussed. Based on typical measured carrier lifetimes and the available literature data on carrier capture cross sections, we estimate that the residual acceptor concentration in CdZnTe detector crystals is much lower than widely thought, about 1011 cm-3 instead of 1015 cm-3. The deleterious effects of structural defects within single crystals are also discussed. We also provide a brief overview of the progress in CdZnTe crystal growth and device fabrication technologies aiming at reducing the concentration of the detrimental defects and improving CdZnTe detector performance.