This paper presents a mathematical model of a machine tool rotary table with backlash to describe the dynamic behavior of the mechanical system and the motion controller. The accuracy of this model is verified by experiments. The steady-state vibration under different conditions is simulated to investigate its mechanism and change rule. The results show that the steady-state vibration is attributed to the alternate impact of transmission components. Based on the different performances of the steady-state vibration for different control gains and different motion directions, the concept of stability region in the plane of control gains is presented. In the critical region, the steady-state vibration only occurs when the table moves toward backlash. The complex contact regimes may lead to a significant increase in the amplitude of the steady-state vibration. Besides, the influences of the load and the magnitude of backlash on the steady-state vibration and the stability region are also discussed.