To enhance the adaptability of the passive co-tracking system in complex environment and improve the tracking accuracy, an adaptive filtering algorithm based on observation noise estimation is proposed. In the proposed framework, the Sage-Husa filtering technique with an improved noise estimator was adopted in UKF algorithm. The improved Sage-Husa algorithm utilized a dynamic statistic of the residual vectors to get better estimation accuracy and stability. In addition, a matching strategy based on a self-correlation portion of the covariance was implemented, which helped prevent the filtering divergence caused by the non-positive noise variance. The experimental results showed that the proposed algorithm can improve the tracking accuracy and robustness effectively.