Instability in DOA manifold ambiguity resolution

We discuss the instability conditions of direction-of-arrival (DOA) manifold ambiguity resolution for uncorrelated Gaussian sources and nonuniform linear antenna arrays. Manifold ambiguity is associated with linear dependence amongst the points on the array manifold (the «steering vectors») where the number of sources is less than the number of sensors, or in the more general case, amongst the points on the co-array manifold. In our previous papers, we have demonstrated that such ambiguity renders subspace-based DOA estimation techniques (such as MUSIC) useless, but does not necessarily imply that the scenario is nonidentifiable. In those identifiable cases of manifold ambiguity, we have proposed a new fitting algorithm to identify the true DOAs from the superset of ambiguous DOA estimates generated by MUSIC. In this paper, we present analytic evidence that the stochastic stability of this identification technique (with respect to the finite sample size) depends on the precise scenario parameters, and may become unstable. We present simulation results that support the analytic predictions.