This study tested if children with specific language impairment (SLI) or children with specific reading disability (SRD) have abnormal brain responses to sounds. We tested 6- to 12-year-old children with SLI (N = 19), children with SRD (N = 55), and age-matched controls (N = 36) for their passive auditory event-related potentials (ERPs) to tones, rapid tones, vowels and consonant-vowels. Thirty-eight percent of the children with SLI or SRD had less typical passive auditory ERPs in the N1-P2 window to sounds in general, rather than to tones, rapid tones, vowels or consonant-vowels specifically. The ERPs of these children were significantly 'flatter' in the N1-P2 region than normal. All the children with flatter ERPs in the N1-P2 region had poor non-word reading. A subgroup of these poor non-word readers also had poor non-word repetition. These findings support the hypothesis that impaired auditory processing is a causal risk factor for both SLI and SRD.