Evidence suggests that living in a socioeconomically deprived neighborhood is associated with worse health. Yet most research relies on cross-sectional data, which implicitly ignore variation in longer-term exposure that may be more consequential for health. Using data from the 1970 to 2011 waves of the Panel Study of Income Dynamics merged with census data on respondents’ neighborhoods (N = 1,757), this study estimates a marginal structural model with inverse probability of treatment and censoring weights to examine: (1) whether cumulative exposure to neighborhood disadvantage from birth through age 17 affects self-rated health in early adulthood, and (2) the extent to which variation in such exposure helps to explain racial disparities therein. Findings reveal that prolonged exposure to neighborhood disadvantage throughout childhood and adolescence is strikingly more common among nonwhite versus white respondents and is associated with significantly greater odds of experiencing an incidence of fair or poor health in early adulthood.