Objective This study aimed to examine the association between change in employment participation for a 12‐month period and quality of life among individuals with colorectal cancer compared with general population controls. Methods This was a prospective, registry‐based study that enrolled middle‐aged (45‐64 years) residents of Queensland, Australia, who were in the paid workforce, and newly diagnosed with colorectal cancer. Participants completed structured telephone interviews at 6 and 12 months after diagnosis assessing quality of life and employment status (“retired/ceased work,” “increased work,” “decreased work,” and “maintained work”). Survivors were matched on demographic and occupation characteristics in a 1:2 ratio with individuals from the general population who had participated in both Wave 10 (2010) and 11 (2011) of the Household, Income and Labour Dynamics in Australia survey. Results Almost half (66/148, 45%) of colorectal cancer survivors ceased or decreased work during the study period, compared with 27% in the control group (79/295, P = .001). Physical and mental well‐being did not fluctuate over time in the general population. However, there were significant improvements in physical well‐being, functional well‐being, and overall quality of life during the study period for participants with colorectal cancer. At 12 months postdiagnosis, participants with colorectal cancer who maintained or increased work had significantly better functional well‐being and overall quality of life compared with those who decreased work or retired. Conclusions A diagnosis of colorectal cancer often impairs the ability of a person to maintain work. The impairments are predominantly physical and functional. Interventions to assist with occupational rehabilitation should be trialed.