Viruses are ubiquitous, abundant and play an important role in all ecosystems. Here, we advance understanding of coastal sediment viruses by exploring links in the composition and abundance of sediment viromes to environmental stressors and sediment bacterial communities. We collected sediment from contaminated and reference sites in Sydney Harbour and used metagenomics to analyse viral community composition. The proportion of phages at contaminated sites was significantly greater than phages at reference sites, whereas eukaryotic viruses were relatively more abundant at reference sites. We observed shifts in viral and bacterial composition between contaminated and reference sites of a similar magnitude. Models based on sediment characteristics revealed that total organic carbon in the sediments explained most of the environmental stress-related variation in the viral dataset. Our results suggest that the presence of anthropogenic contaminants in coastal sediments could be influencing viral community composition with potential consequences for associated hosts and the environment.