Diffusion MRI offers an in vivo probe of tissue microstructure and is commonly used in clinical MRI studies of the brain and body. Physical phantoms play an essential role in its validation, ranging from serving as a reference for calibrating and testing new diffusion MRI sequences and protocols to providing ground-truth for validating biophysical models to extract microstructural features from the diffusion MRI signal. While most often water is used in diffusion MRI phantoms, a gel has a higher viscosity, resulting in less motion and higher similarity to biological tissue, and may thereby be potentially advantageous to use in diffusion MRI phantoms. For that, a good understanding of the water diffusion in gels is needed. Hence, this chapter aims to provide an overview of the diffusion properties of gels, discuss potential (dis)advantages (e.g. compared to water), and illustrate how they can serve as useful diffusion phantoms. In particular, we characterized the diffusion in a 1% agarose gel and found that it behaves as a Gaussian medium with the diffusion coefficient D being independent of the agarose concentration (up to 3%), thereby closely resembling water. We also show that for other gels, glycerin and gelatin-based gels, D decreases with concentration, indicating non-Gaussian diffusion. Finally, we review several applications of agarose gels used in diffusion phantoms.