A new algorithm named the likelihood-based iteration square-root cubature Kalman filter (LISRCKF) is provided in this study. The LISRCKF inherits the virtues of the square-root cubature Kalman filter (SRCKF), which uses the cubature rule-based numerical integration method to calculate the mean and square root of covariance for the non-linear random function. The LISRCKF involves the use of the iterative measurement update and the use of the latest measurement, and the iteration termination criterion based on maximum likelihood is introduced in the measurement update. The LISRCKF algorithm is applied to the state estimation for re-entry ballistic target with unknown ballistic coefficient. Its performance is compared against that of the unscented Kalman filter and SRCKF. Moreover, the suitable choice of iteration number is studied; iteration number 5 is the most appropriate for the LISRCKF algorithm. Simulation results indicate that the LISRCKF algorithm has the features of short run time and fast convergence rate; the advantage in robustness is also demonstrated through the numerical simulation, and it is an effective state estimation method.