Pragmatic language impairments are nearly universal in autism spectrum disorders (ASD). Discourse requires that we monitor information that is shared or mutually known, called “common ground.” While many studies have examined the role of Theory of Mind (ToM) in such impairments, few have examined working memory (WM). Common ground impairments in ASD could reflect limitations in both WM and ToM. This study explored common ground use in youth ages 8–17 years with high‐functioning ASD (n = 13) and typical development (n = 22); groups did not differ on age, gender, IQ, or standardized language. We tracked participants' eye movements while they performed a discourse task in which some information was known only to the participant (e.g., was privileged; a manipulation of ToM). In addition, the amount of privileged information varied (a manipulation of WM). All participants were slower to fixate the target when considering privileged information, and this effect was greatest during high WM load trials. Further, the ASD group was more likely to fixate competing (non‐target) shapes. Predictors of fixation patterns included ASD symptomatology, language ability, ToM, and WM. Groups did not differ in ToM. Individuals with better WM fixated the target more rapidly, suggesting an association between WM capacity and efficient discourse. In addition to ToM knowledge, WM capacity constrains common ground representation and impacts pragmatic skills in ASD. Social impairments in ASD are thus associated with WM capacity, such that deficits in domain‐general, nonsocial processes such as WM exert an influence during complex social interactions. Autism Res 2016, 9: 1340–1352. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.