Chatter affects the surface topography and functional performance of work pieces significantly. The surface topography of work pieces is multi-scale, and the characteristics of different levels of the surface topography are closely connected to the different functional performance of the work piece. The relationship between chatter vibration and surface micro-topography is complicated and not specified. By investigating and understanding this relationship clearly, the manufacturing process can be directed to be controlled more actively and accurately, which helps complete the product with expected surface topography and functional performance. This paper aims to reveal the effect of chatter on the surface micro-topography of gears in grinding. Grinding processes considering different machining states and surface topographies of gears under each process were analyzed comprehensively. The following findings were observed. First, chatter causes significant increase of the tooth flank surface roughness in low frequency and increase of the profile roughness, whereas in a different manner in the different gear flank directions. Second, the influence of chatter mainly concentrates on certain frequency bands of the surface topography, and the effect of chatter on the 3D surface topography is within a frequency range. Third, chatter vibration with its multi-frequency-band characteristics shows a multi-scale influence on the work piece surface topography. The possible mechanisms for the formation of these effects were also discussed.