The total least square method based on singular value decomposition for fitting straight line and plane surface has been developed to deal with the straightness calibration problem. Different from the ordinary least square method only taking into account the error of the dependent variable, total least square method considers the errors of all the variables in a symmetrical way. However, in practice, it is difficult to choose an optimal method for the variable errors of measurement data in an asymmetric way. This article presents an improved calibration method for straightness error of a coordinate measuring machine. The proposed method, named as improved total least square, could fit straight line and plane surface when the variables are in an asymmetric way. In improved total least square method, weight matrices with parameter set between the independent and dependent variables are introduced to augmented matrix. A procedure is developed to determine the parameter . Numerical cases and measurement experiment are given to prove the performance of improved total least square method.