One defines a non-homogeneous space (X,μ) as a metric space equipped with a non-doubling measure μ so that the volume of the ball with center x, radius r has an upper bound of the form rn for some n>0. The aim of this paper is to study the boundedness of Calderón-Zygmund singular integral operators T on various function spaces on (X,μ) such as the Hardy spaces, the Lp spaces, and the regularized BMO spaces. This article thus extends the work of X. Tolsa (Math. Ann. 319:89-149, 2011) on the non-homogeneous space (ℝn,μ) to the setting of a general non-homogeneous space (X,μ). Our framework of the non-homogeneous space (X,μ) is similar to that of Hytönen (2011) and we are able to obtain quite a few properties similar to those of Calderón-Zygmund operators on doubling spaces such as the weak type (1,1) estimate, boundedness from Hardy space into L1, boundedness from L∞ into the regularized BMO, and an interpolation theorem. Furthermore, we prove that the dual space of the Hardy space is the regularized BMO space, obtain a Calderón-Zygmund decomposition on the non-homogeneous space (X,μ), and use this decomposition to show the boundedness of the maximal operators in the form of a Cotlar inequality as well as the boundedness of commutators of Calderón-Zygmund operators and BMO functions.