This article focuses on the extent to which violence against family doctors in England is experienced in gendered terms. It draws on data from two studies: a postal survey of 1,300 general practitioners (GPs) (62% response rate) and in‐depth interviews with 26 doctors who have been assaulted or threatened; and 13 focus groups with primary care teams and 19 in‐depth interviews with GPs who had expressed an interest in the topic of violence against doctors. Most GPs, regardless of gender, reported receiving verbal abuse over the last two years, often interpreted as a consequence of declining deference to professionals, while actual physical assaults and threats were much rarer and more likely to be reported by men. Overall, women GPs were much more likely to express concern about violence and to take personal precautions, although younger male GPs working in inner‐city practices also had high levels of concern. The study shows how some aspects of family doctors’ work has been organised on gendered lines and how these contribute to the differences in experience of violence. We suggest that the increasing proportion of women among family doctors may have implications for these, often tacit, organisational routines.