Adult neuroimaging studies have demonstrated dissociable neural activation patterns in the visual cortex in response to letters (Latin alphabet) and numbers (Arabic numerals), which suggest a strong experiential influence of reading and mathematics on the human visual system. Here, developmental trajectories in the event‐related potential (ERP) patterns evoked by visual processing of letters, numbers, and false fonts were examined in four different age groups (7‐, 10‐, 15‐year‐olds, and young adults). The 15‐year‐olds and adults showed greater neural sensitivity to letters over numbers in the left visual cortex and the reverse pattern in the right visual cortex, extending previous findings in adults to teenagers. In marked contrast, 7‐ and 10‐year‐olds did not show this dissociable neural pattern. Furthermore, the contrast of familiar stimuli (letters or numbers) versus unfamiliar ones (false fonts) showed stark ERP differences between the younger (7‐ and 10‐year‐olds) and the older (15‐year‐olds and adults) participants. These results suggest that both coarse (familiar versus unfamiliar) and fine (letters versus numbers) tuning for letters and numbers continue throughout childhood and early adolescence, demonstrating a profound impact of uniquely human cultural inventions on visual cognition and its development. Hemispheric dissociation between visual processing of letters and numbers (that is observable in adults) is nearly absent in 7‐ and 10‐year‐olds and starts to emerge around 15 years of age. The results suggest a protracted development of the visual cortex influenced by reading and mathematics.