Aims To estimate physical activity trajectories for people who quit smoking, and compare them to what would have been expected had smoking continued. Design, Setting and Participants A total of 5115 participants in the Coronary Artery Risk Development in Young Adults Study (CARDIA) study, a population‐based study of African American and European American people recruited at age 18–30 years in 1985/6 and followed over 25 years. Measurements Physical activity was self‐reported during clinical examinations at baseline (1985/6) and at years 2, 5, 7, 10, 15, 20 and 25 (2010/11); smoking status was reported each year (at examinations or by telephone, and imputed where missing). We used mixed linear models to estimate trajectories of physical activity under varying smoking conditions, with adjustment for participant characteristics and secular trends. Findings We found significant interactions by race/sex (P = 0.02 for the interaction with cumulative years of smoking), hence we investigated the subgroups separately. Increasing years of smoking were associated with a decline in physical activity in black and white women and black men [e.g. coefficient for 10 years of smoking: −0.14; 95% confidence interval (CI) = −0.20 to −0.07, P < 0.001 for white women]. An increase in physical activity was associated with years since smoking cessation in white men (coefficient 0.06; 95% CI = 0 to 0.13, P = 0.05). The physical activity trajectory for people who quit diverged progressively towards higher physical activity from the expected trajectory had smoking continued. For example, physical activity was 34% higher (95% CI = 18 to 52%; P < 0.001) for white women 10 years after stopping compared with continuing smoking for those 10 years (P = 0.21 for race/sex differences). Conclusions Smokers who quit have progressively higher levels of physical activity in the years after quitting compared with continuing smokers.