A quadrotor is a kind of vertical takeoff and landing unmanned aerial rotorcraft which has been attracting wide interest around the world. However, small-scale quadrotors usually suffer from modeling error and external disturbance which are severe challenges for the control system design. To this end, the disturbance-observer-based (DOB) control strategy is applied to reject the performance degradation caused by the above negative factors. To further improve the trajectory tracking precision for an indoor quadrotor suffering from modeling error and external disturbance, a linear dual DOB (LDDOB) trajectory tracking control scheme is developed in this paper. Considering the practical implementation in engineering, the LDDOB position control approach is developed based on the linear quadrotor model. In addition, regarding to the non-minimum phase characteristic of the translational motion model, a modified version of a PID position controller is utilized to attenuate the undershoot of the closed-loop system in the initial response. Extensive numerical simulations show the effectiveness of the proposed trajectory tracking control scheme.