Objectives Human biologists have documented variability in reproductive maturation, fertility, and cancer risk related to developmental conditions. Yet no previous studies have directly examined the impact of prenatal and postnatal energetic environments on sex steroids in infancy, a critical period for hypothalamic–pituitary–gonadal axis development. Thus, we examined the impact of maternal characteristics, birth size, and feeding practices on fecal sex steroid production in a longitudinal sample of 31 American infants followed from 2 weeks to 12 months of age. Methods Maternal characteristics and birth size were collected at study enrollment, infant diet was assessed through weekly 24‐h food diaries, and anthropometrics were measured weekly. Fecal estradiol and testosterone levels were assessed weekly using validated microassay RIA techniques. Mixed models were used to test for associations between maternal and birth characteristics, feeding practices, and sex steroids across the first year of life. Formal mediation analysis examined whether the relationship between infant feeding and hormone levels was mediated by infant size. Results Maternal and birth characteristics had persistent effects on fecal sex steroid levels, with taller maternal height and larger birth size associated with lower estradiol levels in girls and higher testosterone levels in boys. Infant diet was also associated with sex steroid levels independently of infant size. Formula feeding was associated with higher estradiol levels in boys and girls and with higher testosterone in girls. Conclusion These results suggest that markers of early energy availability influence sex hormone levels with potential long‐term consequences for reproductive development and function. Am. J. Hum. Biol., 2013. © 2013 Wiley Periodicals, Inc.