The translocator protein (TSPO) is a target for the development of neuroinflammation imaging agents. Clinical translation of TSPO PET ligands, such as [11C]DPA-713, has been hampered by the presence of a common polymorphism (A147T TSPO), at which all second-generation TSPO ligands lose affinity. Little is known about what drives binding at A147T compared to WT TSPO. This study aimed to identify moieties in DPA-713, and related derivatives, that influence binding at A147T compared to WT TSPO. We found changes to the nitrogen position and number in the heterocyclic core influences affinity to WT and A147T to a similar degree. Hydrogen bonding groups in molecules with an indole core improve binding at A147T compared to WT, a strategy that generated compounds that possess up to ten-times greater affinity for A147T. These results should inform the future design of compounds that bind both A147T and WT TSPO for use in neuroinflammation imaging.