A recursive robust least-squares (RLS) estimator is newly proposed for time-varying linear systems with a noise-corrupted measurement matrix. Analysing the properties of the conventional least-squares (LS) estimator for uncertain systems reaches an inspirational result that the LS estimates contains both scale-factor and bias errors. The scale-factor error is caused by the auto-correlation of the stochastic parametric uncertainties in the measurement matrix, and the bias error comes from the correlation between the uncertain stochastic parameters and measurement noises. On the basis of this observation, the RLS estimation problem is reformulated as finding a compensation strategy of these errors. It is shown that the magnitudes of weighted errors in nominal LS estimates can be approximated using the known statistical information on the stochastic parametric uncertainty. Therefore if the existence condition is satisfied, a recursive RLS solution is readily derived by introducing the proper error compensation method to the nominal LS estimator. The suboptimality of the proposed approach is assessed in the sense of unbiasedness. Furthermore, it is pointed out that the proposed estimator can be reduced to the existing robust Kalman filter under certain conditions. A direct frequency estimation problem is provided to show that the proposed estimator can be an excellent choice for the actual estimator design problem in the presence of stochastic parametric uncertainties.