During the cocaine epidemic of the 1980s and early 1990s, many expressed fears that children with intrauterine cocaine exposure (IUCE) would grow up to be unusually violent. The present study examines the relationship of caregiver reports of school‐age children's aggressive behavior with IUCE and postnatal exposure to violence. Respondents were 140 low‐income, primarily African American children, ages 8–11, and each child's current primary caregiver from a longitudinal study evaluating potential long term sequelae of IUCE. Multiple regression analyses were used to investigate the independent and interactive effects of level of IUCE (None (n = 69), Lighter (n = 47), Heavier (n =  24)) and exposure to violence (Violence Exposure Scale for Children‐Revised) on aggressive behavior (Child Behavior Checklist), while also controlling for other intrauterine substance exposures and additional contextual factors. Children's self‐reported exposure to violence was significantly positively associated with caregivers’ reports of aggressive behavior (β = 2.17, P = .05), as was concurrent caregiver's psychiatric distress (β = .15, P = .003). However, neither IUCE nor its interaction with exposure to violence showed a significant association with aggressive behavior. Findings suggest the importance of postnatal social environment rather than IUCE in predicting aggressive behavior in childhood. Aggr. Behav. 42:209–221, 2016. © 2015 Wiley Periodicals, Inc.