Fluorescent nanodiamonds (NDs) are strong contenders as bio-labels for life science imaging, diagnostics and therapeutics. Ultimately, for their use in biomedical applications, their size should ideally be <10 nm. Yet, even more critical for their specificity and efficient uptake in cellular systems, is their resilience to aggregation, which is dictated by their colloidal stability in complex, physiological environments. To this end, we characterize small detonation NDs (~5 nm) by examining their surface chemical profiles and stability in solutions of varying ionic strength and pH. Using dynamic light scattering measurements, we demonstrate that small monodisperse ND particles with chemically homogeneous and negatively charged surface profiles are more stable than positive particles under a broad range of simulated biological environments. We show that the colloidal stability of small clusters of both positive and negative detonation NDs is improved by functionalization with bovine serum albumin. Based on these analyses, we propose and describe strategies for enhancing the overall colloidal stability of detonation NDs and their resilience to aggregation. Our findings provide a practical framework towards the reduction in size of the bio-conjugates employed to probe complex biological systems, and the advancement of bio-imaging techniques with minimal perturbation of the molecular trafficking in cellular and organelle systems.