Objective: The association between patterns of change in self-rated health (SRH) and memory trajectories in older adults was examined using a systematic approach. Method: Data from the Health and Retirement Study (n = 6,016) and the English Longitudinal Study of Ageing (n = 734) were analyzed. Individuals were grouped into five categories according to their pattern of change in SRH over 8 years: stable excellent/very good/good, stable fair/poor, improvement, decline, and fluctuating pattern without a trend. Memory was measured using immediate and delayed recall tests. Kruskal–Wallis, chi-squares tests, and linear mixed models were used to examine the association. Results: Different rates of decline in memory can be identified in the different patterns of change in SRH. Those who had a stable excellent/very good/good pattern had the slowest rate of decline. Discussion: Our findings suggest that SRH status and patterns of change could be used as a marker of cognitive decline in prevention screening programs.