Background Emerging work suggests that academic achievement may be influenced by the management of affect as well as through efficient information processing of task demands. In particular, mathematical anxiety has attracted recent attention because of its damaging psychological effects and potential associations with mathematical problem solving and achievement. This study investigated the genetic and environmental factors contributing to the observed differences in the anxiety people feel when confronted with mathematical tasks. In addition, the genetic and environmental mechanisms that link mathematical anxiety with math cognition and general anxiety were also explored. Methods Univariate and multivariate quantitative genetic models were conducted in a sample of 514 12‐year‐old twin siblings. Results Genetic factors accounted for roughly 40% of the variation in mathematical anxiety, with the remaining being accounted for by child‐specific environmental factors. Multivariate genetic analyses suggested that mathematical anxiety was influenced by the genetic and nonfamilial environmental risk factors associated with general anxiety and additional independent genetic influences associated with math‐based problem solving. Conclusions The development of mathematical anxiety may involve not only exposure to negative experiences with mathematics, but also likely involves genetic risks related to both anxiety and math cognition. These results suggest that integrating cognitive and affective domains may be particularly important for mathematics and may extend to other areas of academic achievement.