The purpose of the current study was to describe and explain the views on teaching English Language Learners (ELLs) held by six elementary physical education (PE) teachers in the Midwest region of the United States. Situated in positioning theory, the research approach was descriptive–qualitative. The primary sources of data were face-to-face interviews, and the transcript data were analyzed using NVivo 8 software and constant comparative method. The recurrent themes that emerged from the data were as follows: (a) pedagogical challenges; (b) traumatized; (c) irritations, frustrations, and expectations; and (d) cultural dissonance. These themes reflect the teachers’ positioning that teaching ELLs is difficult and complicated. There is pressing need for PE teacher candidates and practicing teachers to receive professional preparation and development training in implementing culturally and ethnolinguistically relevant pedagogies effectively.