People often display excessive overconfidence when providing interval estimates, which biases decision‐making. Research has investigated the various measures to effectively reduce overconfidence, and the use of warnings has subsequently been considered to have a negligible reduction effect. We demonstrate with two separate experiments that the impact of warnings has to be reviewed in light of dynamic warning designs and cognitive warning process models. In experiment 1, in contrast to previous studies that only used unstructured warnings, we implement a warning incorporating some core elements of a structured warning design based on research in the fields of human factors and ergonomics. Furthermore, accounting for recent developments in the warning literature, we distinguish between static and dynamic warning design. In experiment 2, we examine the effectiveness of different elements of dynamic warnings. We show that a significantly higher reduction of overconfidence can be achieved by combining a structured warning content with a dynamic stimulus change to increase the warning's noticeability. Copyright © 2016 John Wiley & Sons, Ltd.