Objective Determine whether serum phosphate is associated with concurrent cognitive impairment and subsequent cognitive decline in older men independent of demographic covariates and atherosclerotic risk factors. Methods In a prospective study of 5529 men enrolled in the Osteoporotic Fractures in Men study, we measured baseline serum phosphate, baseline cognitive function, and change in cognitive function between baseline and follow‐up exams an average of 4.6 years later using the Modified Mini‐Mental State (3MS) Examination and Trails B. Results There was no association between serum phosphate and odds of cognitive impairment as assessed by baseline 3MS score or risk of cognitive decline as assessed by longitudinal change in 3MS score. Higher baseline serum phosphate was associated with higher odds of poor executive function as assessed by Trails B with fully adjusted odds ratios 1.12 (95% confidence interval: 0.83–1.52), 1.31 (0.97–1.77), and 1.45 (1.08–1.94) for men in the second, third, and fourth versus the bottom quartile (referent group) of serum phosphate (p‐trend 0.007). However, higher phosphate level was not associated with risk of decline in executive function as assessed by longitudinal change in Trails B score with fully adjusted odds ratios 0.94 (95% confidence interval 0.69–1.28), 0.96 (0.70–1.32), and 1.21 (0.89–1.66) for men in the second, third, and fourth versus the bottom quartile (referent group) of serum phosphate (p‐trend 0.22). Conclusions Higher serum phosphate in older men was associated with a higher likelihood of poor executive function, but not with impaired global cognitive function or decline in executive or global cognition. Copyright © 2017 John Wiley & Sons, Ltd.