Rapid detection of landmines and explosive hazards is a critical issue for modern military operations. Due to the varied nature of the objects of interest and the complexity of the surroundings, one approach is to utilize the superior recognition capabilities of the human brain in the detection process. We are developing frameworks and algorithms to process image video data from an RGB camera, mounted on a moving vehicle, and to provide cueing capability for a human-in-the-loop detection system. Feedback from the human operator is embedded into the system memory to aid future detection processes (causal). Due to the inherent variation of different objects of interest terms of color and texture appearance, and the inherent variation and complexity of the surroundings, we introduce a classification algorithm that operates on local decision boundaries. Each decision boundary is learned using the support vector machine (SVM) technique. Given test data, the K-nearest neighbor (KNN) method is used to pre-select the nearest training set to localize the scope of SVM training, giving us the local decision boundary.