Aim The objective of the present study was to verify how socioeconomic and physical/mental health status would be associated with activity limitations by age and sex among older adults, using nationally representative cross‐sectional data in Japan. Methods The present study focused on 8373 older adults aged 65 years or older extracted from the Comprehensive Survey of Living Conditions conducted in 2007 by the Japanese Ministry of Health, Labor and Welfare. Univariate and multiple logistic regression analyses and population‐attributable risk were applied to validate the relationships of socioeconomic and physical/mental health status with activity limitations among the total population, and by age groups (young‐old or old‐old) and sex. Results Mental health showed the highest odds ratio and population‐attributable risk in the total population. In addition, low back pain was associated with activity limitations regardless of age and sex. Other musculoskeletal diseases, such as arthropathy and osteoporosis, were related to activity limitations for women, regardless of age, whereas cardiovascular diseases, including angina pectoris/myocardial infarction and cerebral stroke, were associated with activity limitations for men in any age group. There were no statistically significant correlations between socioeconomic status and activity limitations in any groups. Conclusion Mental health was the most important factor of activity limitations in Japanese older adults. Furthermore, low back pain regardless of age and sex, other musculoskeletal diseases only for women and cardiovascular diseases mainly for men could also be significant risk factors to activity limitations. Therefore, preventive approaches of activity limitations considering sex differences are important for older adults in Japan. Geriatr Gerontol Int 2015; ●●: ●●–●●.