The machine tools are consisted of many parts and most of them are connected by the bolts. Accurate modeling of contact stiffness and damping for bolted joint is crucial in predicting the dynamic performance of machine tools. This paper presents a modified three-dimensional fractal contact model to obtain the stiffness and damping of bolted joint. Topography of the contact surface of bolted joint is fractal featured and determined by fractal parameters. Asperities in microscale are considered as elastic, elastic–plastic, and full plastic deformation. The expand coefficient is introduced to the size-distribution function of asperities. The real contact area, contact stiffness, and damping of the contact surface can be calculated by integrating the microasperities. The relationship of contact stiffness, damping, fractal dimension D, and fractal roughness parameter G can be obtained. Experiments are conducted to verify the efficiency of the proposed model. The results show that the theoretical mode shapes are in good agreement with the experimental mode shapes. The relative errors between the theoretical and experimental natural frequencies are less than 3.33%, which is less than those of the W-K model and L-L model. The presented model can be used to accurately predict the dynamic characteristic of bolted assembly in the machine tools.