Biologically relevant simulations for validating risk models under small-sample conditions

In designing scientific experiments, power analysis is too often given a superficial treatment- choice of sample size is often made based on idealized distributions and simplistic tests that do not reflect the real-world constraints under which the actual data will be collected. We have developed a general Monte Carlo framework for two-group comparisons which samples points from a two-dimensional parameter space and at each point generates simulated datasets which are compared to simulated datasets for a “control group” at a fixed point in the parameter space. Rather than uniformly sampling this parameter space, our algorithm rapidly converges on a contour corresponding to the smallest detectable difference for the sample size of interest. We apply this framework, implemented as an R library called PowerTrip, to directly comparing the performance and sensitivity to sample size of the Gompertz survival model to several other commonly used survival models. We find that the Gompertz mortality model performs approximately as well as the Weibull and the Cox models throughout most of the parameter space, but outperforms the competing models in cases where initial mortality rate (IMR) and rate of acceleration (RoA) change in opposite directions.