Background Viral infections of the central nervous system may have detrimental effects for the developing brain, but the effects of less virulent common infections are unclear. We aim to investigate the impact of common viral infections of early childhood on neuropsychological performance of children at age four. Methods We used cross‐sectional data on 674 children participating at the 4 years of age follow‐up of the Rhea birth cohort in Crete, Greece. Blood levels of IgG antibodies to 10 polyomaviruses (BKPyV, JCPyV, KIPyV, WUPyV, HPyV6, HPyV7, TSPyV, MCPyV, HPyV9, and HPyV10) and four herpesviruses [Epstein–Barr virus (EBV), cytomegalovirus (CMV), herpes simplex virus‐1 (HSV‐1), and herpes simplex virus‐2 (HSV‐2)] were measured using multiplex serology. Child's neuropsychological development at age four was assessed using the McCarthy Scales of Children's Abilities, the Attention‐Deficit Hyperactivity Disorder Test (ADHDT), and the Strengths and Difficulties Questionnaire (SDQ). Multiple linear regression models were used to explore the associations. Results Seroprevalence to polyomaviruses ranged from 21% for HPyV9 to 82% for HPyV10. Seroprevalence for EBV was 53%, for CMV 26%, for HSV‐1 3.6%, and for HSV‐2 1.5%. Children seropositive to ≥8 polyomaviruses had lower score in ADHDT inattention subscale [β = −1.28 (95% CI: −2.56, −0.001)] and lower score in SDQ hyperactivity–inattention subscale [β = −.99 (95% CI: −1.60, −0.37)] versus children seropositive to ≤3 polyomaviruses. Seropositivity to BKPyV, a potential neurotropic virus, was associated with higher score in ADHDT inattention subscale [β = .87 (95% CI: 0.03, 1.71)]. Conclusions These findings suggest that acquisition of polyomaviruses during development may influence behavioral outcomes in early childhood.