Multi-atlas based segmentation-propagation approaches have been shown to obtain accurate parcelation of brain structures. However, this approach requires a large number of manually delineated atlases, which are often not available. We propose a supervised method to build a population specific atlas database, using the publicly available Internet Brain Segmentation Repository (IBSR). The set of atlases grows iteratively as new atlases are added, so that its segmentation capability may be enhanced in the multi-atlas based approach. Using a dataset of 210 MR images of elderly subjects (170 elderly controls, 40 Alzheimer's disease) from the Australian Imaging, Biomarkers and Lifestyle (AIBL) study, 40 MR images were segmented to build a population specific atlas database for the purpose of multi-atlas segmentation-propagation. The population specific atlases were used to segment the elderly population of 210 MR images, and were evaluated in terms of the agreement among the propagated labels. The agreement was measured by using the entropy H of the probability image produced when fused by voting rule and the partial moment μ2 of the histogram. Compared with using IBSR atlases, the population specific atlases obtained a higher agreement when dealing with images of elderly subjects.