People usually respond faster to a visual stimulus when it is immediately preceded by a task‐irrelevant, auditory accessory stimulus (AS). This AS effect occurs even in choice reaction time tasks, despite the fact that the AS carries no information about the correct response. Researchers often assume that the AS effect is mediated by a phasic arousal burst evoked by the AS, but direct evidence for that assumption is lacking. We conducted a pupillometry study to directly test the phasic arousal hypothesis. Participants carried out a demanding choice reaction time task with accessory stimuli occurring on 25% of the trials. Pupil diameter, a common index of arousal, was measured throughout the task. Standard analyses of task performance and pupil diameter showed that participants exhibited the typical AS effect, and that accessory stimuli evoked a reliable early pupil dilation on top of the more protracted dilation associated with the imperative stimulus. Moreover, regression analyses at the single‐trial level showed that variation in reaction times on AS trials was selectively associated with pupil dilation during the early time window within which the AS had an effect, such that particularly large AS‐evoked dilations were associated with especially fast responses. These results provide the first evidence that the AS effect is mediated by AS‐evoked phasic arousal.