Network embedding is an efficient method to learn low-dimensional representations of vertexes in networks since the network structure can be captured and preserved through this process. Unlike shallow models, deep neural network framework is able to capture the highly non-linear network structure. Therefore, it can achieve much better performance in comparison of traditional network embedding methods. However, few attention has been paid to the smoothness of such models, in contrast to numerous research works for image and text fields. Methods without smoothness are not robust enough, which means that slight changes on network may lead dramatic changes on the embedding results. Hence, how to find a smooth deep framework is still an open yet important problem. To this end, in this paper, we propose a Smooth Deep Network Embedding method, namely SmNE, which generates stable and reliable embedding results. Empirically, we conduct experiments on real-world networks. The results show that compared to the state-of-the-art methods, our proposed method can achieve significant gains in several applications.