This paper presents a spatiotemporal framework for estimating single-trial response latencies and amplitudes from evoked response magnetoencephalographic/electroencephalographic data. Spatial and temporal bases are employed to capture the aspects of the evoked response that are consistent across trials. Trial amplitudes are assumed independent but have the same underlying normal distribution with unknown mean and variance. The trial latency is assumed to be deterministic but unknown. We assume that the noise is spatially correlated with unknown covariance matrix. We introduce a generalized expectation-maximization algorithm called Trial Variability in Amplitude and Latency (TriViAL) that computes the maximum likelihood (ML) estimates of the amplitudes, latencies, basis coefficients, and noise covariance matrix. The proposed approach also performs ML source localization by scanning the TriViAL algorithm over spatial bases corresponding to different locations on the cortical surface. Source locations are identified as the locations corresponding to large likelihood values. The effectiveness of the TriViAL algorithm is demonstrated using simulated data and human evoked response experiments. The localization performance is validated using tactile stimulation of the finger. The efficacy of the algorithm in estimating latency variability is shown using the known dependence of the M100 auditory response latency to stimulus tone frequency. We also demonstrate that estimation of response amplitude is improved when latency is included in the signal model.