In this paper, we first present a detailed study on the trade-off between the computational complexity (directly related to the power consumption) and classification accuracy for a number of classifiers for classifying normal and abnormal electrocardiograms (ECGs). In our analysis, we consider the spectral energy of the constituent waves of the ECG as the discriminative feature. Starting with the exhaustive exploration of single heartbeat-based classification to ascertain the complexity-accuracy trade-off in different classification algorithms, we then extend our study for multiple heartbeat-based classification. We use data available in Physionet as well as samples from Southampton General Hospital Cardiology Department for our investigation. Our primary conclusion is that a classifier based on linear discriminant analysis (LDA) achieves comparable level of accuracy to the best performing support vector machine classifiers with advantage of significantly reduced computational complexity. Subsequently, we propose an ultra low-power circuit implementation of the LDA classifier that could be integrated with the ECG sensor node enabling on-body normal and abnormal ECG classification. The simulated circuit is synthesized at 130 nm technology and occupies 0.70 mm2 of silicon area (0.979 mm2 after place and route) while it consumes 182.94 nW @ 1.08 V, estimated with Synopsys PrimeTime when operating at 1 KHz. These results clearly demonstrate the potential for low-power implementation of the proposed design.