We studied attention effects on the integration of written and spoken syllables in fluent adult readers by using event‐related brain potentials. Auditory consonant‐vowel syllables, including consonant and frequency changes, were presented in synchrony with written syllables or their scrambled images. Participants responded to longer‐duration auditory targets (auditory attention), longer‐duration visual targets (visual attention), longer‐duration auditory and visual targets (audiovisual attention), or counted backwards mentally. We found larger negative responses for spoken consonant changes when they were accompanied by written syllables than when they were accompanied by scrambled text. This effect occurred at an early latency (∼ 140 ms) during audiovisual attention and later (∼ 200 ms) during visual attention. Thus, audiovisual attention boosts the integration of speech sounds and letters.