Children and adults show behavioral evidence of psychological overlap between their early, non‐symbolic numerical concepts and their later‐developing symbolic numerical concepts. An open question is to what extent the common cognitive signatures observed between different numerical notations are coupled with physical overlap in neural processes. We show that from 8 years of age, regions of the intraparietal sulcus (IPS) that exhibit a numerical ratio effect during non‐symbolic numerical judgments also show a semantic distance effect for symbolic number words. In both children and adults, the IPS showed a semantic distance effect during magnitude judgments of number words (i.e. larger/smaller number) but not for magnitude judgments of object words (i.e. larger/smaller object size). The results provide novel evidence of conceptual overlap between neural representations of symbolic and non‐symbolic numerical values that cannot be explained by a general process, and present the first demonstration of an early‐developing dissociation between number words and object words in the human brain. When making magnitude judgments, there is a dissociation between number words and object words that emerges by at least 8 years of age in children: the IPS processes number words and ventral temporal cortex processes object words. Number words likely dissociate from object words because, as we show, they are rooted in numerosity judgments of physical sets.