Rigidity of thought is considered a main characteristic of persons with Asperger syndrome (AS). This rigidity may explain the poor comprehension of unusual semantic relations, frequently exhibited by persons with AS. Research indicates that such deficiency is related to altered mental lexicon organization, but has never been directly examined. The present study used computational network science tools to compare the mental lexicon structure of persons with AS and matched controls. Persons with AS and matched controls generated free associations, and network tools were used to extract and compare the mental lexicon structure of the two groups. The analysis revealed that persons with AS exhibit a hyper-modular semantic organization: their mental lexicon is more compartmentalized compared to matched controls. We argue that this hyper-modularity may be related to the rigidity of thought which characterizes persons with AS and discuss the clinical and more general cognitive implications of our findings.