Individual defects in crystalline materials can have electronic properties akin to those of isolated trapped atoms or ions. Like atoms, these defect centers can have spin degrees of freedom and and optical transitions that make them an attractive platform for building quantum information technologies. Their spin states might someday be used to store and manipulate quantum information, with photons connecting individual defects into a useful computational network or secure communication system. A major challenge to this vision is the efficiency of the photonic interface, particularly for emitters such as the nitrogen-vacancy (NV) center in diamond, whose optical transitions couple strongly to strain, phonons, or electric fields. This talk will present results on the path toward creating a high-efficiency spin-photon interface for NV centers using fiber-based optical microcavities. These cavities promise a complementary approach to nanophotonic devices, with reduced diamond fabrication requirements and very high quality factors.