Aim To clarify the effectiveness of dietary diversity, calculated by dietary records, on cognitive decline. Methods Data were derived from the National Institute for Longevity Sciences – Longitudinal Study of Aging. Participants comprised 298 men and 272 women aged 60–81 years at baseline (second wave) who participated in the follow‐up study (third to seventh wave) at least once. Cognitive function was assessed with the Mini‐Mental State Examination in all study waves. Dietary diversity was determined using the Quantitative Index for Dietary Diversity based on a 3‐day dietary record in the second wave. Cumulative data among participants with a Mini‐Mental State Examination score >27 in the second wave were analyzed using a generalized estimating equation. Multivariate adjusted odds ratios and 95% confidence intervals for Mini‐Mental State Examination scores ≤27 in each study wave according to a 1 standard deviation (increase), or quartiles of the Quantitative Index for Dietary Diversity at baseline, were adjusted for sex, age, follow‐up time, baseline Mini‐Mental State Examination score, education, body mass index, annual household income, current smoking status, energy intake and disease history. Results Multivariate adjusted odds ratio for a decline in Mini‐Mental State Examination score was 0.79 (95% CI 0.70–0.89; P < 0.001) with a 1 SD increase in dietary diversity score, or 1.00 (reference), 0.99 (95% CI 0.70–1.43), 0.68 (95% CI 0.46–0.99) and 0.56 (95% CI 0.38–0.83) according to the lowest through highest quartiles of dietary diversity score, respectively (trend P = 0.001). Conclusions Daily intake of various kinds of food might be a protective factor against cognitive decline in community‐dwelling Japanese older adults. Geriatr Gerontol Int 2017; 17: 937–944.