Using a stochastic production frontier to model potato production in Bolivia, we quantify the costs of environmental and activity diversification (AD) in the form of efficiency losses and yield forgone. We find that efficiency decreases with the number of fields in a geographical cluster, distance between the dwelling and a particular field, discontinuity between fields, and off‐farm income. However, environmental diversification (ED) is more detrimental than AD. Using spatial analysis of field and household efficiency measures, we assess production vulnerability to climatic shocks and the potential of ED in mitigating shocks. We find important spatial clusters of low and high efficiencies at the field level suggesting that climatic shocks influence efficiency measures. Household‐level efficiency measures exhibit random spatial patterns suggesting that on average households can mitigate the adverse effects of shocks through ED.