Study Objectives: To evaluate the association between self-reported sleep quality and levels of brain β-amyloid (Aβ) burden, and to determine the effect of the apolipoprotein E (APOE) ϵ4 allele on any associations found. Methods: This study is a cross-sectional analysis of 184 cognitively healthy men and women aged over 60 y. We measured sleep quality factors: specifically, sleep duration, latency (time taken to fall asleep), disturbances, efficiency, daytime dysfunction, and overall sleep quality, using the Pittsburgh Sleep Quality Index. All participants underwent Aβ positron emission tomography imaging for the quantification of brain Aβ burden and were APOE genotyped. Linear regression analyses were used to evaluate the relationship between sleep quality factors and brain Aβ burden, adjusting for age, body mass index, cardiovascular disease, and symptoms of depression, with APOE ϵ4 carriage entered as a moderator. Results: Of the sleep factors, longer sleep latency was associated with higher levels of brain Aβ (B = 0.003 [standard error = 0.001], P = 0.02). APOE ϵ4 allele (carrier/noncarrier) did not moderate the relationship between sleep latency and brain Aβ burden. Conclusions: Our findings suggest a relationship between brain Aβ burden and sleep latency, independent of APOE ϵ4 genotype.