Objective Cognitive training offers a promising way to mitigate cognitive deterioration in individuals with mild cognitive impairment (MCI). This randomized control pilot trial examined the effects of Gist Reasoning Training on cognition as compared with a training involving New Learning in a well‐characterized MCI group. Methods Fifty participants with amnestic MCI were randomly assigned to the experimental Gist Training group or an active control New Learning group. Both groups received 8 h of training over a 4‐week period. We compared pre‐training with post‐training changes in cognitive functions between the two training groups. Results The Gist Training group showed higher performance in executive function (strategic control and concept abstraction) and memory span compared with the New Learning group. Conversely, the New Learning group showed gains in memory for details. Conclusion These findings suggest that cognitive training in general yields benefits, and more specifically, training programs that target top–down cognitive functions such as gist reasoning may have a broad impact on improving cognition in MCI. © 2016 The Authors. International Journal of Geriatric Psychiatry Published by John Wiley & Sons Ltd.