It remains unclear whether the N170 of ERPs reflects domain‐specific or domain‐general visual object processing. In this study, we used superimposed images of a face and a Chinese character such that participants' relative expertise for the two object types was either similar (Experiment 1 and 2) or different (Experiment 3). Experiment 1 showed that N170 amplitude was larger when participants attended to the character instead of the face of a face‐character combination. This result was unchanged in Experiment 2, in which task difficulty was selectively increased for the face component of the combined stimuli. Experiment 3 showed that, although this N170 enhancement for attending to characters relative to faces persisted for false characters with recognizable parts, it disappeared for unrecognizable characters. Therefore, N170 amplitude was significantly greater for Chinese characters than for faces presented within a combined image, independent of the relative task difficulty. This result strongly calls N170 face selectivity into question, demonstrating that, contrary to the expectations established by a domain‐specific account, N170 is modulated by expertise.