Objective Cancer is still widely feared and often associated with death. Fatalistic beliefs adversely affect help‐seeking for cancer symptoms and engagement in cancer prevention. This study aims to understand middle‐aged and older adults’ first association with the word ‘cancer’, and their relationship with sociodemographic factors, cancer fear, and cancer information avoidance. Methods We conducted a cross‐sectional survey of 1464 community‐based adults aged 50 to 70 living in England in April 2015. First associations with cancer were measured qualitatively and analysed using content analysis. We used binary logistic regression to analyse associations between the most common first association of cancer and sociodemographic characteristics, cancer fear and cancer information avoidance. Results Cancer was most commonly associated with ‘death’ (26%). Respondents with lower levels of education, living in the Midlands or North of England where cancer mortality is higher, or with close friends or family members with a cancer history, were more likely to associate cancer with death. Cancer fear was significantly associated with death associations, but cancer information avoidance was not. Conclusions Despite improved cancer outcomes, middle‐aged and older adults often associate cancer with death. Further efforts to decrease fatalistic associations in this age group may be needed.