Many studies have shown that children with autism spectrum disorder have some understanding of intentions behind others’ goal-directed actions on objects. It is not clear whether they understand intentions at a high level of abstraction reliant on the context in which the actions occur. This study tested their understanding of others’ prior intentions with typically developing and developmentally delayed children. We replicated Carpenter et al.’s test of the ability to understand prior intentions embedded in the social situation with an additional context of no prior intention. Results showed that when the experimenter’s intention was made known before the demonstration, children without autism spectrum disorder performed not only better than the autism spectrum disorder children but also better than themselves when there was no information about prior intention. No between-condition difference was found in the autism spectrum disorder group. It thus appears that children with autism spectrum disorder have difficulty decoupling intentions from the context of the situation. The present findings, together with previous evidence for the intactness of the ability to understand and to imitate goal-directed actions, suggest that asymmetrical imitation performance occurs at different levels of understanding of intention by children with autism spectrum disorder.