In an era wherein racism is strongly condemned, everyday processes of categorization and othering confront individuals with contradictions that need to be managed. This article discusses how some of the categorization processes that parents of Flemish and Italian origin – living in Flanders (Belgium) – use are based upon negative stereotyping of the other and clash with one’s general self-image of a tolerant and nonracist person. Consequently, the possibility of being stigmatized as racist needs to be managed and avoided. Rather than refraining from excluding and negatively stereotyping the other, individuals tend to develop new strategies. The family context proved to be an interesting context to study this management of the representation of the self, and of the other. An analysis of 27 interviews with Belgian and Italian origin parents in Flanders clarifies how the shifting of labels from the ‘ethnic other’ (Turks and Moroccans) to the ‘religious other’ (Muslims) is used to legitimize exclusion. The discourses reveal an instrumental use of the (re)labeling of the (same) other which offers parents a solution to achieve the same end-results – the exclusion of the other from the family context – without fearing stigmatization as racist.