Research summary: We develop a theory to explain why new outside CEOs can better manage their relationship with the board if they previously served on boards that were more diverse than the focal board. We predict that a new outside CEO's prior experience with more diverse boards not only reduces the likelihood of post‐succession CEO turnover and director turnover, but also improves firm performance. Results from an analysis of 188 outside CEOs in a sample of Fortune 500 companies provide support for our theory. This study contributes to upper echelon theory and research by identifying outside CEOs' prior experience with board diversity as an important aspect of their background that influences a range of major organizational outcomes, including CEO turnover, director turnover, and firm performance. Managerial summary: It is challenging to be a new CEO who comes from outside of the organization. Our study examines why some new outside CEOs fare better than others. We suggest that a positive relationship with the board of directors is a key factor in a new outside CEO's success. A new outside CEO can better manage the relationship with the board if he or she has prior experience working with other demographically diverse boards. In contrast, when the focal board is more diverse than the other boards on which the new CEO previously served, the new CEO tends to struggle in managing his or her relationship with the board, experiencing a higher likelihood of turnover and delivering worse financial performance. Copyright © 2015 John Wiley & Sons, Ltd.