The recovery of structure from motion in real time over extended areas demands methods that mitigate the effects of computational complexity and arithmetical inconsistency. In this paper, we develop SCISM, an algorithm based on relative frame bundle adjustment, which splits the recovered map of 3D landmarks and keyframes poses so that the camera can continue to grow and explore a local map in real time while, at the same time, a bulk map is optimized in the background. By temporarily excluding certain measurements, it ensures that both maps are consistent, and by using the relative frame representation, new results from the bulk process can update the local process without disturbance. The paper first shows how to apply this representation to the parallel tracking and mapping (PTAM) method, a real-time bundle adjuster, and compares results obtained using global and relative frames. It then explains the relative representation's use in SCISM and describes an implementation using PTAM. The paper provides evidence of the algorithm's real-time operation in outdoor scenes, and includes comparison with a more conventional submapping approach.