Congenital prosopagnosia (CP) is a selective difficulty in recognizing familiar faces that is present from birth. There is mounting evidence for a familial factor in CP, possibly due to a simple autosomal inheritance pattern. However, potential candidate genes remain to be established, and the question whether genetically based CP is a single trait, or a cluster of related subtypes differing in the pattern of impairments to specific components of the face-processing system, remains unanswered. In addition, since the great majority of so far described cases with CP were adult at the time of investigation, it remains unknown which specific aspects of face processing are impaired in small children with CP. Here we present the first study that specifically addresses these questions by elucidating the specific mechanisms underlying face-recognition impairments in seven individuals with CP (aged 4-87 years) belonging to four generations of the same family. Our results indicate that genetically based CP is not a single trait but a cluster of related subtypes, since the pattern of impairments to specific components of the face-processing system varies in individuals belonging to the same family. In addition, we show that the heterogeneity of the cognitive profile in CP with respect to specific aspects of face processing is apparent from early childhood.