This paper proposes a new method to design a reinforcement learning-based integrated kinematic and dynamic tracking control algorithm for a non-holonomic wheeled mobile robot without knowledge of the system’s drift tracking dynamics. The actor critic structure in the control scheme uses only one neural network to reduce computational cost and storage resources. A novel tuning law for a single neural network is designed to learn an online solution of a tracking Hamilton–Jacobi–Isaacs (HJI) equation. The HJI solution is used to approximate an H optimal tracking performance index function and an intelligent tracking control law in the case of the worst disturbance. The laws guarantee closed-loop stability in real time. The convergence and stability of the overall system are proved by Lyapunov techniques. The simulation results on a non-linear system and wheeled mobile robot verify the effectiveness of the proposed controller.