Aim Studies of the association between body composition, both body fat and body muscle, and cognitive function are rarely reported. The aim of the present study was to determine the association between a wide range of body composition measures with cognitive function in older adults. Methods A total of 2322 Malaysian older adults aged 60 years and older were recruited using multistage random sampling in a population‐based cross‐sectional study. Out of 2322 older adults recruited, 2309 (48% men) completed assessments on cognitive function and body composition. Cognitive functions were assessed using the Malay version of the Mini‐Mental State Examination, the Bahasa Malaysia version of Montreal Cognitive Assessment, Digit Span Test, Digit Symbol Test and Rey Auditory Verbal Learning Test. Body composition included body mass index, mid‐upper arm circumference, waist circumference, calf circumference, waist‐to‐hip ratio, percentage body fat and skeletal muscle mass. Results The association between body composition and cognitive functions was analyzed using multiple linear regression. After adjustment for age, education years, hypertension, hypercholesterolemia, diabetes mellitus, depression, smoking status and alcohol consumption, we found that calf circumference appeared as a significant predictor for all cognitive tests among both men and women (P < 0.05), except for the Rey Auditory Verbal Learning Test. Waist‐to‐hip ratio was detected as a significant predictor for all cognitive tests among women (P < 0.05), but was only a significant predictor for the Bahasa Malaysia version of Montreal Cognitive Assessment among men (P < 0.05). Conclusions These results suggest that there is a need to maintain muscle mass and lower adipose tissue among older adults for optimal cognitive function. Geriatr Gerontol Int 2017; 17: 554–560.