Auditory distractions can have serious consequences in critical situations such as driving. Mobile phones, radios, media players, and information devices that interpret and produce speech are increasingly common in vehicles, but the threats to visual attention are not yet fully understood. In three experiments, we found that most speech tasks had relatively small adverse effects on the detection of a briefly presented target among distractors across a 60° subarea of the visual field. Although there was a little impact on detectability, moderately difficult speech tasks slowed responding relative to silence. Our most demanding condition—generating and speaking a word beginning with the last letter of another word—had the greatest effects on accuracy and latency, with responding slowed by about 900 ms. An impairment of this magnitude presents a significant threat to safe driving and calls into question the belief that hands‐free voice‐controlled devices are the answer to the problem of driver distraction. Copyright © 2013 John Wiley & Sons, Ltd.