Purpose Colorectal cancer (CRC) incidence and mortality in the United States have steadily declined since the 1980s, but racial and socioeconomic disparities remain. The influence of geographic factors is poorly understood and may be affected by evolving insurance coverage and screening test uptake. We characterized temporal trends in the association between geographic and sociodemographic factors and CRC outcomes. Methods We used the 1973‐2010 SEER‐Medicare files to identify patients aged ≥65 years with and without CRC. Beneficiary residential ZIP codes were used to extract local‐level data. We constructed multivariable logistic regression models for CRC incidence and mortality using geographic and sociodemographic variables in 4 time periods: (1) 1973‐1997; (2) 1998‐2001; (3) 2002‐2006; and (4) 2007‐2010. Findings We analyzed 1,093,758 records, including 336,321 CRC cases. Compared to urban residence, small rural residence was strongly associated with increased CRC incidence (OR 1.50, 95% CI: 1.43‐1.57) and mortality (OR 1.35, 95% CI: 1.26‐1.45) in 1973‐1997, but the associations diminished by 2007‐2010 (OR 1.09, 95% CI: 1.04‐1.15 for incidence; OR 1.10, 95% CI: 1.01‐1.20 for mortality). The disparity between blacks and whites increased over time for both incidence (OR 1.09, 95% CI: 1.05‐1.13 in 1973‐1997 vs OR 1.32, 95% CI: 1.27‐1.37 in 2007‐2010) and mortality (OR 1.22, 95% CI: 1.16‐1.28 in 1973‐1997 vs OR 1.34, 95% CI: 1.26‐1.42 in 2007‐2010). High socioeconomic status was associated with greater incidence and mortality in 1973‐1997, but it became protective after 1998. Conclusions Although disparities persist among Medicare beneficiaries, the relationship between geographic and sociodemographic factors and CRC incidence and mortality has evolved over time.