Children with autism spectrum disorders (ASD) and children with Williams syndrome (WS) show divergent social phenotypes, but also several similarities in their socio‐cognitive deficits. Cross‐syndrome direct comparisons could lead to a better understanding of mechanisms that determine deficits in social cognition in the two syndromes. A fundamental factor for social cognition is the ability to understand and predict others' actions (e.g. what action is being done and why it is being done when observing a goal‐related act). Here we compared the understanding of others' actions in children with ASD, WS and in children with typical development. Comprehension of what motor act was being done and of why it was being done was assessed with or without contextual cueing using a computer‐based task. The results showed that what understanding was impaired in the WS group, but not in the ASD group, which showed mental‐age appropriate performance. Why understanding was impaired in both experimental groups. Autism Res 2014, ●●: ●●–●●. © 2014 International Society for Autism Research, Wiley Periodicals, Inc.