Objective To examine the association between neuropsychiatric symptoms (NPS) with self‐ and caregiver‐rated Quality of Life (QoL) for patients with Alzheimer's disease (AD) during a 5‐year follow‐up. Methods The ALSOVA 5‐year follow‐up study included, at baseline, 236 patients with either very mild (Clinical Dementia Rating Scale (CDR) 0.5), or mild (CDR 1) AD, together with their caregivers from three Finnish hospital districts. QoL was evaluated using patient self‐reported, and caregiver‐rated, QoL in AD (QoL‐AD) scores. NPS were assessed using the Neuropsychiatric Inventory (NPI), and AD severity was evaluated using the CDR, with cognition tested by the mini‐mental state examination. The performance of daily activities was assessed using the Alzheimer's Disease Cooperative Study–Activities of Daily Living Inventory. Results Over the 5‐year follow‐up period, patient self‐reported QoL‐AD scores did not change significantly (p = 0.245), despite increases in their NPS. However, caregiver‐rated patient QoL‐AD scores declined significantly (p ≤ 0.001), as total NPI scores increased during follow‐up. No NPS at baseline, and only apathy at follow‐up, correlated significantly (p = 0.007) with patient self‐rated QoL‐AD scores. Caregiver‐rated patient QoL‐AD scores correlated significantly with most NPS, especially (p ≤ 0.001) apathy, agitation, anxiety, irritability, depression, and delusions at baseline, and delusions, hallucinations, apathy, appetite disturbances, and anxiety during follow‐up. Conclusions Patient rated QoL‐AD scores are an unreliable tool with which to evaluate the success of therapy for NPS. Instead, caregiver‐rated scores for patients correlated well with NPI scores, and health care professionals in the clinic should preferentially use these. Copyright © 2017 John Wiley & Sons, Ltd.