Regression lines: more than meets the eye

In science and engineering, we often want to see how well a linear relation between two sets of data describes how they are related. Given such a regression line, we can interpolate data and (less reliably) extrapolate them or provide an explanatory relation between the variables. Fitting a regression line lets us quantify relationships and produce a line that is much better than an eyeball fit. In the physical sciences and engineering, one variable is often measured more accurately than the other; they are then called independent and dependent variables, respectively. Social scientists and economists commonly use the terms control and response variables, while engineers often make such analyses as part of parametric identification. We summarize some methods for linear regression that go beyond the simple straight-line least-squares fits commonly used. We discuss several topics on how regression-line fitting is related to maximum likelihood, to errors in the variables, to a large scatter in errors, and to quantifying deviations from linearity arising from intrinsic scatter in the variables. The more sophisticated methods often require significantly more computing than the simple algorithms. We discuss, for example, the bootstrap technique, which uses resampling from the given data to estimate uncertainty in the regression lines.