The sources of knowledge that individuals use to make similarity judgments between words are thought to tap underlying phonological representations. We examined the effects of perceptual similarity between stimuli on deaf childrens' ability to make judgments about the phonological similarity between words at 3 levels of linguistic structure (syllable, rhyme, and phoneme). Manipulation of stimulus contrasts (acoustic, visual/orthographic, tactile/motoric) allowed a finer-grained estimate of the sources of knowledge that deaf individuals use to make similarity judgments between words. The results showed that the ability to make syllable-, rhyme-, and phoneme-level judgments was not tied to "phonological" facilitation when these conditions are contrasted. These findings are inconsistent with long-held assumptions of "functional" equivalence between "heard" and "seen" speech in the development of phonological representations in deaf learners. We argue that previous studies reporting evidence for phonological effects in similarity judgments have failed to sufficiently control for alternative sources of sensory information, namely, visual and tactile/motoric.