Cross-lingual Entity Alignment (CEA) aims at identifying entities with their counterparts in different language knowledge graphs. Knowledge embedding alignment plays an important role in CEA due to its advantages of easy implementation and run-time robustness. However, existing embedding alignment methods haven't considered the problem of embedding distribution alignment which refers to the alignment of spatial shapes of embedding spaces. To this end, we present a new Adversarial Knowledge Embedding framework (AKE for short) that jointly learns the representation, mapping and adversarial modules in an end-to-end manner. By reducing the discrepancy of embedding distributions, AKE can approximately preserve an isomorphism between source and target embeddings. In addition, we introduce two new orthogonality constraints into mapping to obtain the self-consistency and numerical stability of transformation. Experiments on real-world datasets demonstrate that our method significantly outperforms state-of-the-art baselines.