Physical activity enhances cognitive performance, yet individual variability in its effectiveness limits its widespread therapeutic application. Genetic differences might be one source of this variation. For example, carriers of the methionine-specifying (Met) allele of the brain-derived neurotrophic factor (BDNF) Val66Met polymorphism have reduced secretion of BDNF and poorer memory, yet physical activity increases BDNF levels. To determine whether the BDNF polymorphism moderated an association of physical activity with cognitive functioning among 1,032 midlife volunteers (mean age = 44.59 years), we evaluated participants’ performance on a battery of tests assessing memory, learning, and executive processes, and evaluated their physical activity with the Paffenbarger Physical Activity Questionnaire. BDNF genotype interacted robustly with physical activity to affect working memory, but not other areas of cognitive functioning. In particular, greater levels of physical activity offset a deleterious effect of the Met allele on working memory performance. These findings suggest that physical activity can modulate domain-specific genetic (BDNF) effects on cognition.