According to different authors, computer games not only teach contents and skills, but also do so in a more efficient manner, allowing long‐lasting learning. However, there is still little consensus on this matter as different studies put their educational benefits into question, especially when used without instructional support. An empirical study was conducted to measure the effect of the educational game The Conference Interpreter on L2 vocabulary acquisition and perceived learning gains, as compared with a non‐gaming tool which replicated its contents. The results of pre‐, post‐ and delayed tests showed that students that had access to the contents via the video game performed statistically better in the short run, found the materials more appealing and believed their vocabulary skills had developed further than those in the control group. However, the regression model showed that the actual enjoyment of the game seemed to have no effect on the students' learning outcomes, neither according to their own estimation nor as determined by testing. Of greater importance seemed to be extrinsic motivation, ie, their desire to play based upon expected learning gains, prior knowledge of tested L2 vocabulary and perceived difficulty of the educational contents.