Task switch costs often show an asymmetry, with switch costs being larger when switching from a difficult task to an easier task. This asymmetry has been explained by difficult tasks being represented more strongly and consequently requiring more inhibition prior to switching to the easier task. The present study shows that switch cost asymmetries observed in arithmetic tasks (addition vs. subtraction) do not depend on task difficulty: Switch costs of similar magnitudes were obtained when participants were presented with unsolvable pseudo-equations that did not differ in task difficulty. Further experiments showed that neither task switch costs nor switch cost asymmetries were due to perceptual factors (e.g., perceptual priming effects). These findings suggest that asymmetrical switch costs can be brought about by the association of some tasks with greater difficulty than others. Moreover, the finding that asymmetrical switch costs were observed (1) in the absence of a task switch proper and (2) without differences in task difficulty, suggests that present theories of task switch costs and switch cost asymmetries are in important ways incomplete and need to be modified.