Aim Little is known about longitudinal changes of body composition measured by dual‐energy X‐ray absorptiometry (DXA) in middle‐aged and elderly individuals. We evaluated longitudinal changes of body composition, and age and sex differences in appendicular skeletal muscle mass. Methods Participants were 1454 community‐dwelling Japanese men and women aged 40–79 years. Body composition at baseline and 6‐year follow up was measured by DXA. Results Fat increased significantly in men of all ages, and in women aged in their 40s and 50s. Among men, arm lean tissue mass (LTM) changed by 0.9%, −0.5%, −1.4% and −3.7%, respectively, for the 40s to the 70s, and decreased significantly in the 60s and 70s. Leg LTM in men changed by −0.4%, −1.3%, −1.7% and −3.9%, respectively, and decreased significantly from the 50s to the 70s. Compared with the preceding age groups, significant differences were observed between the 60s and 70s in arm and leg LTM change in men. Among women, arm LTM changed by 0.7%, 0.2%, 1.6% and −1.5%, respectively, which was significant in the 60s and 70s. Leg LTM decreased significantly in all age groups of women by −2.0%, −2.8%, −2.4% and −3.9%, respectively. With respect to sex differences, leg LTM loss rates were significantly higher in women than men at the 40s and 50s. Conclusions Longitudinal data suggest that arm and leg LTM decreased markedly in men in their 70s, and leg LTM had already decreased in women in their 40s. Geriatr Gerontol Int 2014; 14: 354–361.