The ability to design and interpret controlled experiments is an important scientific process skill and a common objective of science standards. Numerous intervention studies have investigated how the control‐of‐variables‐strategy (CVS) can be introduced to students. However, a meta‐analysis of 72 intervention studies found that the opportunity to train CVS skills with hands‐on tasks (g = 0.59) did not lead to better acquisition of CVS relative to interventions without a hands‐on component (g = 0.74). We conducted an intervention study in which we investigated the differential effects of hands‐on and paper‐and‐pencil training tasks on 161 eighth‐grade students’ achievement. CVS was demonstrated to all students before they were grouped into a hands‐on or a paper‐and‐pencil training condition. In both training conditions, students designed and interpreted experiments about which variables influence the force of electromagnets. Students in the hands‐on group interacted with physical equipment while students in the paper‐and‐pencil group planned experiments using sketches and interpreted the outcome of experiments presented in photographs. We found no general advantage or disadvantage of hands‐on tasks, as both groups did equally well on CVS and content knowledge tests. However, hands‐on students outperformed paper‐and‐pencil students on a hands‐on test identical to the training tasks, whereas the paper‐and‐pencil students outperformed hands‐on students on a science fair poster evaluation task similar to the paper‐and‐pencil training. In summary, students learned task‐specific procedural knowledge, but they did not acquire a deeper conceptual understanding of CVS or the content domain as a function of type of training. Implications for instruction and assessment are discussed. © 2016 Wiley Periodicals, Inc. J Res Sci Teach 53:980–1002, 2016