We introduce a novel algorithm to address the challenges in magnetic resonance (MR) spectroscopic imaging. In contrast to classical sequential data processing schemes, the proposed method combines the reconstruction and postprocessing steps into a unified algorithm. This integrated approach enables us to inject a range of prior information into the data processing scheme, thus constraining the reconstructions. We use high resolution, 3-D estimate of the magnetic field inhomogeneity map to generate an accurate forward model, while a high resolution estimate of the fat/water boundary is used to minimize spectral leakage artifacts. We parameterize the spectrum at each voxel as a sparse linear combination of spikes and polynomials to capture the metabolite and baseline components, respectively. The constrained model makes the problem better conditioned in regions with significant field inhomogeneity, thus enabling the recovery even in regions with high field map variations. To exploit the high resolution MR information, we formulate the problem as an anatomically constrained total variation optimization scheme on a grid with the same spacing as the magnetic resonance imaging data. We analyze the performance of the proposed scheme using phantom and human subjects. Quantitative and qualitative comparisons indicate a significant improvement in spectral quality and lower leakage artifacts.