A new approach to automatic music transcription for music with two timbres from a monaural sound source is proposed in this paper. The system is mainly composed of two parts, a fundamental frequency detector and a timbre discriminator. In the fundamental frequency detector, the short time Fourier transform (STFT) and a peak detection algorithm are used. By combining the relative magnitude of the fundamental frequency and its harmonics, a characteristic timbre vector is computed. The timbre discrimination is done by the classification of the timbre vectors with the support vector machine (SVM). Particularly, by designing the training and classification procedure in SVM, the mixed-timbre signal can also be classified in this system. Using a small database of polyphonic music of two timbres as the testing input, a 73% hit rate is achieved by this system.