Aim To investigate the effects of 5 years of physical exercise on functional parameters among older Japanese adults who carried out water‐ or land‐based exercise. Methods We retrospectively investigated data from 5707 medical examinations and enrolled 77 older adults into the study. Eligible participants had to be aged ≥60 years, and engaged in water‐based exercise (n = 38) or a combination of water‐ and land‐based exercise (n = 39) for at least 80% of their total exercise time for over 5 years at our fitness center. In statistical analysis, a two‐way repeated‐measures analysis of variance was carried out to examine the effects over time and by exercise type, and the changes in each parameter over 5 years were also compared between the two groups. Results We found significant main effects and an interaction between time and exercise type for gait speed, with an early decline in the combined exercise group, as well as significant main effects of time, showing a functional decline in grip strength, one‐leg standing time and step/height ratio in both exercise types at the 5‐year follow up. The 5‐year changes in each parameter did not differ between the two groups despite the frequency of exercise, even though we found a negative correlation between changes in one‐leg standing time and total amount of water‐based exercise. Conclusion Contrary to expectations, these results suggest that regular engagement in water‐based exercise, even combined with land‐based exercise, might have poor long‐term benefits for maintaining physical performance in older adults. Geriatr Gerontol Int 2017; ••: ••–••.