The social intelligence hypothesis (SIH) posits that within-group interactions drive cognitive evolution, but it has received equivocal support. We argue the SIH overlooks a major component of social life: interactions with conspecific outsiders. Competition for vital resources means conspecific outsiders present myriad threats and opportunities in all animal taxa across the social spectrum (from individuals to groups). We detail cognitive challenges generated by conspecific outsiders, arguing these select for ‘Napoleonic’ intelligence; explain potential influences on the SIH; and highlight important considerations when empirically testing these ideas. Including interactions with conspecific outsiders may substantially improve our understanding of cognitive evolution.