We conduct novel analysis to assess the impact of load balancing on the rate coverage performance in a millimeter wave (mmWave) cellular heterogeneous network (HetNet) with a macro-tier and a micro- tier. To ensure the generality of our analysis, we adopt the Poisson point process for the location of base stations (BSs) and user equipments, the line- of-sight ball model for mmWave links, the sectored antenna model for key antenna array characteristics, and Nakagami-m fading for wireless channels. We first analyze the loads of macro- and micro-tier BSs, based on which we derive a new expression for the rate coverage probability of the network. Through numerical results, we demonstrate the correctness of our analysis. In addition, we thoroughly examine the impact of load balancing and various network parameters on the rate coverage probability in various scenarios, offering valuable guidelines into the design of practical mmWave HetNets.