To obtain good performance in the conventional energy regeneration system which has worse dynamic actuator performance than traditional orifice control systems, an energy regeneration system that employs a proportional throttle valve and a proportional directional valve to avail a hydraulic motor controlling the boom is proposed. The control model is constructed and the control properties are analyzed. Based on the proposed energy regeneration system, the new operation mode of the boom system with an energy regeneration system is characterized by the status of the joystick and the pressure of the boom cylinder. Considering the performance under the bad control in the low rotational speed, the total power loss of the hydraulic motor and the generator, a dynamic-working-points control strategy of the generator is discussed. Then, as the displacement and velocity sensors cannot be easily assembled in a hydraulic excavator, a compound control that consists of throttle control and volume control is presented. Finally, a test rig is constructed, the recovery efficiency of the proposed energy regeneration system is approximately 35% and it is also shown that the proposed energy regeneration system had better control performance than the conventional energy regeneration system.