Core-shell colloids composed of a dielectric core surrounded by a metal shell show geometric cavity resonances with optical properties that are distinctly different than those of the collective plasmon modes of the metal shell. We use finite-difference time domain calculations on silica colloids with a core diameter of 456 nm, surrounded by a 38 nm thick Au shell, to study the temporal evolution of the mode field intensity inside the cavity upon pulsed excitation. Calculations using Mie theory and the T-matrix method are used to analytically determine the dipolar cavity resonance spectrum, which is found superimposed on the broad collective dipolar plasmonic resonance modes. We characterize resonance wavelength and linewidth in terms of a geometric mode confined inside the cavity. Cavity linewidth can be controlled by metal shell thickness and quality factors Q≫150 are observed. Due to the small cavity mode volume V=0.2(λ/n)3, a Purcell factor as high as P=54 is calculated. Introducing shape anisotropy lifts the cavity mode degeneracy, yielding blue- and redshifted longitudinal and transverse resonant modes, respectively. The relatively large volume over which the field enhancement is observed in these spherical and anisotropic metal shell cavities, combined with cavity quality factors that are much higher than that of the collective plasmonic modes, makes them attractive for application in nanoscale light sources, sensors, or lasers.