A detailed theoretical analysis of stability is presented for a semiconductor laser in an external cavity. The limits of stable operation are determined as a function of the external cavity parameters and the linewidth enhancement factor α. Instability is related to jumps of the laser frequency between external cavity modes (frequency bistability) or to feedback-induced intensity pulsations due to the carrier density dependence of the refractive index. The limit of bistability is derived from the steady-state solutions of the rate equations and the intensity pulsation limit is obtained from a small-signal analysis. This analysis also gives the location of zeros in the system determinant and the resulting FM noise spectrum. For practical applications we emphasize the determination of the stable tuning range for the phase in the external cavity and the classification of the possible types of instability for various feedback levels.