Background It is unclear whether brain white matter hyperintensities (WMHI) causes or is a result of late life depression. We used the Framingham Heart Study offspring to examine whether indices of brain aging are related to incident depression in the elderly. Methods The Center for Epidemiologic Studies Depression Scale (CES‐D) was administered along with a brain MRI scan at baseline and was re‐administered (n = 1212) at an average 6.6 + 0.6 year follow‐up. The outcomes (i) change in CES‐D scores from baseline; (ii) depression defined as CES‐D ≥16; (iii) severe depression defined as CES‐D ≥21; and (iv) CES‐D cutoff scores and/or on antidepressant were used. Results Among those who did not have depression at baseline, 9.1% (n = 110) developed depression, 4.0% (n = 48) developed severe depressive symptoms, and 11.1% (n = 135) were put on antidepressants. When depressive symptoms only was the outcome, we found that baseline WMHI was positively associated with change in CES‐D scores and that those with an extensive WMHI at baseline had a high risk of developing severe depressive symptoms; the relationship was strengthened in the absence of cardiovascular diseases. In contrast, when depressive symptoms or taking antidepressant was the outcome, larger total cerebral brain volume and temporal lobe brain volume, but not WMHI, were negatively associated with the development of depression. Conclusions Brain WMHI is a probable risk factor for vascular depression in the elderly. The depression outcomes with and without antidepressant were related to different brain pathologies. Copyright © 2016 John Wiley & Sons, Ltd.