Aim Despite increased interest in an obesity paradox (i.e. a survival advantage of being obese), evidence remains sparse in Japanese populations. We aimed to verify this phenomenon among community‐dwelling older adults in Japan. Methods Older adults aged 65–84 years randomly chosen from all 74 municipalities in Shizuoka Prefecture completed questionnaires including body mass index information. Participants were followed from 1999 to 2009. Following World Health Organization guidelines, participants were classified using an appropriate body mass index for Asian populations as follows: <18.5 kg/m2 (underweight), 18.5–23.0 kg/m2 (normal weight), 23.0–27.5 kg/m2 (overweight) and ≥27.5 kg/m2 (obesity). We estimated hazard ratios and their 95% confidence intervals for all‐cause mortality, controlling for sex, age, smoking status, alcohol consumption, physical activity, hypertension and diabetes mellitus. Results Compared with normal‐weight participants, overweight/obese participants tended to have lower hazard ratios; the multivariate hazard ratios (95% confidence interval) were 0.86 (0.62–1.19) for obesity, 0.83 (0.73–0.94) for overweight and 1.60 (1.40–1.82) for underweight. In subgroup analyses by sex and age, the hazard ratios tended to be lower among obese men, albeit not significantly; hazard ratios (95% confidence interval) were 0.56 (0.25–1.27) in men aged 65–74 years, and 0.78 (0.41–1.45) in men aged 75–84 years. Conclusions The present study provides evidence of a conservative obesity paradox among older Japanese people, using the appropriate body mass index cut‐off points for Asian populations. In particular, obese older men tend to have a lower risk of all‐cause mortality. Geriatr Gerontol Int 2017; 17: 1257–1264.