In the number magnitude decision task ("Is the number bigger/smaller than 5?"), response to a target (e.g., 3) is faster following a masked prime congruent with the target (e.g., 1) than it is following an incongruent prime (e.g., 9). This category congruence effect has been reported to be "interference-dominant" relative to a neutral prime (e.g., the # sign, the number 5) on the basis of the analysis of mean response time (RT). Using RT distribution analysis as well as mean RTs, we identified two bases for this pattern. One relates to the choice of neutral baseline: The # prime, unlike the digit prime, does not factor in the cost of perceptual transition between the prime and target, and therefore underestimates facilitation and overestimates the interference effect. The second basis of the interference-dominant pattern is a disproportionate slowdown of congruent trials in the slow RT bins. Furthermore, this slowdown is greater for primes that had been used as targets than it is with "novel" primes that have not been responded to as targets. We interpret the results as suggesting that the category congruence effect has two components with different time courses - one based on stimulus-response mapping, and the other on semantic categorization.