Cognitive impairment is common among children born very preterm (VPT), yet little is known about how this risk changes over time. To examine this issue, a regional cohort of 110 VPT (≤ 32 weeks gestation) and 113 full‐term (FT) born children was prospectively assessed at ages 4, 6, 9, and 12 years using the Wechsler Preschool and Primary Scale of Intelligence–Revised and then Wechsler Intelligence Scale for Children, 4th ed. At all ages, VPT children obtained lower scores than their FT born peers (p < .001). Growth curve modeling revealed stable cognitive trajectories across both groups. Neonatal white matter abnormalities and family socioeconomic adversity additively predicted cognitive risk. Despite some intraindividual variability, cognitive functioning of typically developing and high‐risk VPT children was stable and influenced by early neurological development and family rearing context.