Decades of research on perception and prediction of randomness led us to speculate that the various response tendencies observed in these studies might manifest in multi‐trial discrimination tasks used in medical education. By re‐analyzing data from a previously published study in which 46 physicians and medical trainees judged 234 pediatric ankle radiographs, we show that (i) response tendencies can be differentially induced when individuals receive uniquely ordered sequences and (ii) response patterns consistent with win‐stay/lose‐shift and win‐shift/lose‐stay heuristics can be predicted from stimulus alternation rates and marginal distributions. Our results illustrate the importance of carefully arranging trials when studying discrimination and when using discrimination tasks to teach or to assess learners' skill levels. We call into question the wisdom of designing studies that present uniquely ordered stimulus sequences and discuss alternatives. Copyright © 2013 John Wiley & Sons, Ltd.