Background: To date evidence of the relationship between cognition and Aβ amyloid during the early stages of Alzheimer's Disease (AD) has been inconsistent. This study aimed to describe the nature and magnitude of the relationship between Aβ amyloid and cognitive performance of individuals without dementia. Methods: Composite cognitive measures were developed from the Australian Imaging Biomarkers and Lifestyle study neuropsychological test battery using data from 768 healthy older adults and 133 adults with mild cognitive impairment (MCI). A subgroup of this sample (174 healthy, 53 MCI) underwent neuroimaging for Aβ amyloid. Results: Within the MCI group individuals with high Aβ amyloid showed selective impairment for memory compared with those with low Aβ amyloid; however, this difference was not evident in the healthy group. Conclusions: The current findings provide further evidence of the relationship between Aβ amyloid and cognition, with memory impairment being the primary symptom of the underlying disease during the prodromal phases of AD.