Hearing loss is associated with anecdotal reports of fatigue during periods of sustained listening. However, few studies have attempted to measure changes in arousal, as a potential marker of fatigue, over the course of a sustained listening task. The present study aimed to examine subjective, behavioral, and physiological indices of listening‐related fatigue. Twenty‐four normal‐hearing young adults performed a speech‐picture verification task in different signal‐to‐noise ratios (SNRs) while their pupil size was monitored and response times recorded. Growth curve analysis revealed a significantly steeper linear decrease in pupil size in the more challenging SNR, but only in the second half of the trial block. Changes in pupil dynamics over the course of the more challenging listening condition block suggest a reduction in physiological arousal. Behavioral and self‐report measures did not reveal any differences between listening conditions. This is the first study to show reduced physiological arousal during a sustained listening task, with changes over time consistent with the onset of fatigue.