Background There has been little qualitative analysis of the experience of stigma, social comparisons and conception of identity among adults with intellectual disabilities (ID). This study aimed to develop an understanding of how adults with intellectual disabilities experience their own disability, and any implications relating to self‐esteem, stigma and social interactions. Materials and Methods Fifteen adults with intellectual disabilities were interviewed using semi‐structured, open‐ended questions regarding disability, social interactions and self‐esteem. Interviews were analysed independently by two researchers using interpretive phenomenological analysis. Results Three major themes emerged during analysis, exploring pressure on participants to behave in a socially normative way, tendency to produce personal definitions of disability and consistently limited knowledge of and discomfort around common disability terminology. Conclusions Participants' clearly experienced feelings of difference, despite a lack of articulation. Limited understanding of both terminology and conceptualization of disability status could negatively impact self‐esteem, person‐centred actions and political movement.