We present the design and fabrication of nanobeam photonic crystal cavities in single crystal diamond for applications in cavity quantum electrodynamics. First, we describe three-dimensional finite-difference time-domain simulations of a high quality factor (Q∼106) and small mode volume [V∼0.5(λ/n)3] cavity whose resonance corresponds to the zero-phonon transition (637 nm) of the nitrogen-vacancy color center in diamond. This high Q/V structure, which would allow for strong light-matter interaction, is achieved by gradually tapering the size of the photonic crystal holes between the defect center and the mirror regions of the nanobeam. Next, we demonstrate two different focused ion beam (FIB) fabrication strategies to generate thin diamond membranes and nanobeam photonic crystal resonators from a bulk crystal. These approaches include a diamond crystal “side-milling” procedure as well as an application of the “lift-out” technique used in transmission electron microscopy sample preparation. Finally, we discuss certain aspects of the FIB fabrication routine that are a challenge to the realization of the high Q/V designs.