Because the collision avoidance function is indispensable for providing safe and easy operation of human-operated robotic systems, this paper deals with the collision avoidance control for a human-operated mobile robot in unknown environments. A typical four-wheeled mobile robot with infrared distance sensors for detecting obstacles is considered. The robot cannot move in an arbitrary direction owing to a nonholonomic constraint. Therefore, we propose a simple control approach in which a human operator’s control input is modified in real time to satisfy the nonholonomic constraint and avoid collision with obstacles. The proposed controller has steering- and brake-like functions that are adjusted according to the distance sensor information. The stability of the proposed control system is analyzed with a linear model. The effectiveness of the proposed method is confirmed by experiments in which several operators control the robot in an environment with obstacles.