A data-adaptive algorithm is presented for the selection of the basis functions and training data used in classifier design with application to sensing mine-like targets with a side-scan sonar. Automatic detection of mine-like targets using side-scan sonar imagery is complicated by the variability of the target, clutter, and background signatures. Specifically, the strong dependence of the data on environmental conditions vitiates the assumption that one may perform a priori algorithm training using separate side-scan sonar data collected previously. In this paper, a novel active-learning algorithm is developed based on kernel classifiers with the goal of enhancing detection/classification of mines without requiring an a priori training set. It is assumed that divers and/or unmanned underwater vehicles (UUVs) may be used to determine the binary labels (target/clutter) of a small number of signatures from a given side-scan collection. These sets of signatures and associated labels are then used to train a kernel-based algorithm with which the remaining side-scan signatures are classified. Information-theoretic concepts are used to adaptively construct the form of the kernel classifier and to determine which signatures and associated labels would be most informative in the context of algorithm training. Using measured side-looking sonar data, the authors demonstrate that the number of signatures for which labels are required (via diver/UUV) is often small relative to the total number of potential targets in a given image. This procedure designs the detection/classification algorithm on the observed data itself without requiring a priori training data and also allows adaptation as environmental conditions change.