Vaccine refusal occurs for a variety of reasons. In this article we examine vaccine refusals that are made on conscientious grounds; that is, for religious, moral, or philosophical reasons. We focus on two questions: first, whether people should be entitled to conscientiously object to vaccination against contagious diseases (either for themselves or for their children); second, if so, to what constraints or requirements should conscientious objection (CO) to vaccination be subject. To address these questions, we consider an analogy between CO to vaccination and CO to military service. We argue that conscientious objectors to vaccination should make an appropriate contribution to society in lieu of being vaccinated. The contribution to be made will depend on the severity of the relevant disease(s), its morbidity, and also the likelihood that vaccine refusal will lead to harm. In particular, the contribution required will depend on whether the rate of CO in a given population threatens herd immunity to the disease in question: for severe or highly contagious diseases, if the population rate of CO becomes high enough to threaten herd immunity, the requirements for CO could become so onerous that CO, though in principle permissible, would be de facto impermissible.