Research summary: Separating the individual from the social effects of incentives has been challenging because of the possibility of synergies in team production. We observe a unique natural experiment in a South Korean e‐commerce company in which a switch from pay‐for‐performance to fixed (but different) salaries took place in a staggered and effectively random manner across employees. In this case, social and individual effects perspectives make opposing predictions, enabling a critical test. We find evidence consistent with social effects of incentives, particularly as predicted by goal framing theory. The results have implications for the design of incentives to foster collaboration, organizational learning, and organizational performance. Managerial summary: Managers often neglect the deeper hypothesis behind pay‐for‐performance schemes—that people primarily care about how much they are individually paid. An opposing school of thought contends that incentives have social effects too—that individuals care about not only what they receive but also what their peers receive. It is difficult to say whether individual or social effects would be more salient in a context, without a proper experiment with randomization. We exploit a rare opportunity provided by a company that changed its incentive system in a random order, thus unintentionally creating a natural experiment. The results strongly validate the existence of social effects of incentives, but also make the general case for the opportunity to learn from experimenting with organization design in a systematic manner. Copyright © 2017 John Wiley & Sons, Ltd.