Objective: This study investigates whether the rate of decline in older persons’ hearing status is associated with the rate of decrease in their psychosocial health and explores moderation by baseline hearing status, health-related factors, and sociodemographic factors. Method: Multilevel analyses were applied to data of 1,178 older participants from the Longitudinal Aging Study Amsterdam (LASA), covering 3 to 7 years of follow-up. Results: Faster decrease in speech-in-noise recognition was significantly associated with more increase in loneliness for persons with a moderate baseline speech-in-noise recognition (emotional and social loneliness) and for persons who recently lost their partner (emotional loneliness). No relationship was found with depression. Discussion: The results indicate that faster hearing decline results in more increase in loneliness in specific subgroups of older persons: in persons with an already impaired hearing and in widow(er)s. Monitoring older persons’ hearing seems important and may be a relevant starting point for targeted loneliness prevention efforts.