Objectives The main goals of this study are to 1) quantitatively measure skin, hair, and iris pigmentation in a diverse sample of individuals, 2) describe variation within and between these samples, and 3) demonstrate how quantitative measures can facilitate genotype‐phenotype association tests. Materials and Methods We quantitatively characterize skin, hair, and iris pigmentation using the Melanin (M) Index (skin) and CIELab values (hair) in 1,450 individuals who self‐identify as African American, East Asian, European, Hispanic, or South Asian. We also quantify iris pigmentation in a subset of these individuals using CIELab values from high‐resolution iris photographs. We compare mean skin M index and hair and iris CIELab values among populations using ANOVA and MANOVA respectively and test for genotype‐phenotype associations in the European sample. Results All five populations are significantly different for skin (P <2 × 10−16) and hair color (P <2 × 10−16). Our quantitative analysis of iris and hair pigmentation reinforces the continuous, rather than discrete, nature of these traits. We confirm the association of three loci (rs16891982, rs12203592, and rs12913832) with skin pigmentation and four loci (rs12913832, rs12203592, rs12896399, and rs16891982) with hair pigmentation. Interestingly, the derived rs12203592 T allele located within the IRF4 gene is associated with lighter skin but darker hair color. Discussion The quantitative methods used here provide a fine‐scale assessment of pigmentation phenotype and facilitate genotype‐phenotype associations, even with relatively small sample sizes. This represents an important expansion of current investigations into pigmentation phenotype and associated genetic variation by including non‐European and admixed populations. Am J Phys Anthropol 160:570–581, 2016. © 2015 Wiley Periodicals, Inc.