While private clouds are still widely adopted due primarily to privacy and security reasons, they are often less resilient with fluctuating workloads compared to public clouds. Workload surges in private clouds can be dealt with by offloading some workload/jobs to public clouds; this is referred to as cloud bursting. Although the dynamic use of public clouds is claimed to be cost efficient, the actual realization of such cost efficiency is highly dependent on judicious scheduling decisions. In this paper, we present Cost Efficient Cloud Bursting Scheduler (CECBS) as a new scheduling framework that optimizes cost efficiency while preserving privacy by taking advantage of benefits of each of two cloud types. In particular, CECBS schedules jobs taking into account (1) public cloud pricing policy (billing cycle), (2) privacy of data/job and (3) local electricity rates for private clouds. Based on simulation results obtained from real workload traces, CECBS achieves 20% cost savings on average compared with costs of Resource Management Service (RMS).