This paper investigates the design of a composite nonlinear feedback (CNF) control law for an overhead crane servo system to improve the transient performance of both displacement tracking of the trolley and anti-sway of the payload. To address the property of underactuation of the overhead crane system, a novel nonlinear function of the CNF control law is specifically proposed to compromise the tracking performance of the trolley and the anti-sway performance of the payload. The performance improvement in both tracking of the trolley and anti-sway of the payload is illustrated with a complete comparison between the CNF control method and the trajectory planning method, which has been proposed in recent literature. The simulation results show that this well-tuned CNF control law can significantly shorten the settling time of the trolley displacement tracking and reduce the sway of the payload.