Aim The present study aimed to examine whether nutrient intakes predicted cognitive decline among elderly patients with diabetes mellitus. Methods This study evaluated data from a 6‐year prospective follow up of 237 elderly patients (aged ≥65 years) with diabetes mellitus, and the associations of baseline nutrient intakes with cognitive decline. Cognitive decline was defined as a ≥2‐point decrease in the Mini‐Mental State Examination (MMSE) score. Intakes of food and nutrients were assessed using a validated food frequency questionnaire, and were compared between patients with cognitive decline and intact cognition. Analysis of covariance and logistic regression analysis were used to compare the changes in the MMSE score during the follow up among intake tertile groups for each nutrient. Results Compared with men with intact cognition, the men with cognitive decline had lower baseline intakes of calcium, vitamin A, vitamin B2, pantothenate, soluble fiber, green vegetables and milk. However, no significant associations between cognitive decline and nutrient intakes were observed among women. After adjusting for age, body mass index, glycated hemoglobin levels, history of severe hypoglycemia, previous stroke and baseline MMSE score, we found that cognitive decline was significantly associated with low intakes of carotene, vitamin B2, pantothenate, calcium and green vegetables. Multiple logistic regression analysis showed that intakes of nutrients and green vegetables predicted cognitive decline after adjusting for age, body mass index, glycated hemoglobin levels, baseline MMSE score, and incident stroke during the follow up. Conclusions These findings suggest that sufficient intakes of carotene, vitamin B2, pantothenate, calcium and vegetables could help prevent cognitive decline among elderly men with diabetes mellitus. Geriatr Gerontol Int 2017; 17: 1168–1175.