The Geneva Convention on the Status of Refugees is central to scholarship on refugee and asylum issues. It is the primary basis upon which asylum seekers make their claims to the majority of host states today and, as a key text of the human rights framework, has come to be associated with the very idea of a universalised rights‐bearing human being. Yet British asylum policy today is characterized by efforts to limit access to the right to asylum. Many scholars believe this is because asylum seekers today are different, in character and number, to previous cohorts of applicants. This article goes back to the founding of the refugee rights regime and investigates the exclusions of colonized peoples from access to the right to asylum. Using Chimni's concept of the “myth of difference”, the article demonstrates that asylum seekers have long existed outside of Europe, and that their exclusion from international rights has been both longstanding and intentional. This historical sociology suggests that the basis for critical work on the issue of asylum policy today must be one which takes colonial histories into account.