Visual selective attention is known to be guided by stimulus‐based (bottom‐up) and goal‐oriented (top‐down) control mechanisms. Recent work has pointed out that selection history (i.e., the bias to prioritize items that have been previously attended) can result in a learning experience that also has a substantial impact on subsequent attention guidance. The present study examined to what extent goal‐oriented top‐down control mechanisms interact with an observer's individual selection history in guiding attention. Selection history was manipulated in a categorization task in a between‐subjects design, where participants learned that either color or shape was the response‐relevant dimension. The impact of this experience was assessed in a compound visual search task with an additional color distractor. Top‐down preparation for each search trial was enabled by a pretrial task cue (Experiment 1) or a fixed, predictable trial sequence (Experiment 2). Reaction times and ERPs served as indicators of attention deployment. Results showed that attention was captured by the color distractor when participants had learned that color predicted the correct response in the categorization learning task, suggesting that a bias for predictive stimulus features had developed. The possibility to prepare for the search task reduced the bias, but could not entirely overrule this selection history effect. In Experiment 3, both tasks were performed in separate sessions, and the bias still persisted. These results indicate that selection history considerably shapes selective attention and continues to do so persistently even when the task allowed for high top‐down control.