Beware λ-truncation! Sample truncation and bias in luminosity calibration using trigonometric parallaxes

The common practice in luminosity calibration of sample truncation according to relative parallax error λ can lead to bias with indirect methods such as reduced parallaxes as well as with direct methods. This bias is not cancelled by the Lutz-Kelker corrections and in fact can be either negative or positive. Making the selection stricter can actually lead to a larger absolute amount of bias and lower accuracy in certain cases. The degree to which this bias is present depends upon whether the sample is more nearly specified by the relative parallax error or by the limiting apparent magnitude when both limits formally apply; when the latter limit dominates it is absent. The difference between the means for the two extreme cases is what is customarily termed the Malmquist bias. However, it is not truly bias, but rather what we call here an offset. For a sample to be effectively magnitude-limited, there is a lower bound imposed on the mean absolute magnitude which depends on the limiting magnitude. If a wide-ranging luminosity relation such as the Wilson-Bappu relation is to be calibrated, some portion of the relation may be magnitude-limited and the rest not. In that case there will be offsets between the different parts of the relation, including the transition region between the two extremes, as well as bias outside the magnitude-limited part. Another, less common, practice is truncation according to weight, specifically with the reduced parallax method. Such truncation can also bias the calibration with one variant of the method. Indeed, the weighting scheme used with that variant introduces bias even without truncation. For calibration it is probably best to use a general maximum likelihood method such as the grid method with a magnitude-limited sample and no limit on relative parallax error. The Malmquist shift could then be applied to obtain an estimate of the volume-limited mean.