Background Depression and anxiety have been related to poorer self‐reported physical functioning over time; however, objective measures of physical function are less frequently examined. This study assessed the 6‐year trajectory of hand‐grip strength and lung function in persons with depressive and/or anxiety disorders. Methods At four waves (baseline, 2, 4, and 6 years) hand‐grip strength and lung function were assessed in 2,480 participants, aged 18–65 years, of the Netherlands Study of Depression and Anxiety. Linear mixed models were used to examine the association between baseline psychiatric status (current and remitted depression and anxiety, healthy controls) and physical function during 6‐year follow‐up, adjusted for sociodemographics, lifestyle, and health indicators. Results Although there were no differences in the rate of decline over time, women with current, but not remitted, depression and anxiety had poorer hand‐grip strength (B = −1.34, P < .001) and poorer lung function (B = −11.91, P =.002) compared to healthy women during the entire 6‐year follow‐up. Associations with depression and anxiety severity measures confirmed dose–response relationships with objective physical function. In men, stronger 6‐year decline of lung function was found in those with current disorders (current diagnosis‐by‐time: B = −11.72, P = .002) and even in those with remitted disorders (remitted diagnosis by time: B = −10.11, P = .04) compared to healthy men. Conclusions Depression and anxiety are associated with consistently poorer hand‐grip strength in women and poorer lung function in women and men over 6 years of time, implicating their long‐lasting impact on physical functioning.