Background:  Sensory over‐responsivity (SOR) affects many individuals with autism spectrum disorders (ASD), often leading to stressful encounters during daily routines. Methods:  This study describes the associations between early SOR symptoms and the longitudinal course of restrictions in family life activities and parenting stress across three time‐points in families raising a child with ASD (n = 174). Covariates were child diagnostic severity, emotional problems, and maternal affective symptoms. At time 1 mean chronological age was 28.5 months. Children were administered the Autism Diagnostic Observation Schedule (ADOS) and Mullen Scales of Early Learning (MSEL). Parents completed the Infant Toddler Sensory Profile (ITSP), Infant‐Toddler Social Emotional Assessment (ITSEA), Beck Anxiety Index (BAI), and the Center for Epidemiologic Studies Depression Inventory (CES‐D) at time 1; and the Parenting Stress Index (PSI) and Family Life Impairment Scale (FLIS) at the three annual time‐points. Results:  Latent Growth Curve Models indicated that higher SOR scores on the ITSP at time 1 were associated with higher initial levels of family life impairment and parenting stress and with a smaller magnitude of change over time. These associations were independent of severity of ADOS social‐communication symptoms, MSEL composite score, ITSEA externalizing and anxiety symptoms, and maternal affective symptoms as measured by the BAI and CES‐D. On average FLIS and PSI did not change over time, however, there was significant individual variability. Concurrently, SOR at time 1 explained 39–45% of the variance in family stress and impairment variables. Conclusions:  An evaluation of SOR should be integrated into the assessment of toddlers with ASD considering their role in family life impairment and stress.