People with autism spectrum disorder (ASD) have difficulties in identifying another person by face and voice. This might contribute considerably to the development of social cognition and interaction difficulties. The characteristics of the voice recognition deficit in ASD are unknown. Here, we used a comprehensive behavioral test battery to systematically investigate voice processing in high‐functioning ASD (n = 16) and typically developed pair‐wise matched controls (n = 16). The ASD group had particular difficulties with discriminating, learning, and recognizing unfamiliar voices, while recognizing famous voices was relatively intact. Tests on acoustic processing abilities showed that the ASD group had a specific deficit in vocal pitch perception that was dissociable from otherwise intact acoustic processing (i.e., musical pitch, musical, and vocal timbre perception). Our results allow a characterization of the voice recognition deficit in ASD: The findings indicate that in high‐functioning ASD, the difficulty to recognize voices is particularly pronounced for learning novel voices and the recognition of unfamiliar peoples’ voices. This pattern might be indicative of difficulties with integrating the acoustic characteristics of the voice into a coherent percept—a function that has been previously associated with voice‐selective regions in the posterior superior temporal sulcus/gyrus of the human brain. Autism Res 2017, 10: 155–168. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.