Using modern samples of known kin, this paper evaluates two fundamental aspects of kinship analysis in archaeological contexts: (1) choice of data (dental metrics or morphology) and (2) analytical approach (multivariate, distance‐based approach or a ‘rare trait’ analysis). Stone dental casts were analysed from 155 female individuals from four sampling locations in Kenya. Of these 155 individuals, only three pairs were close kin: mother–daughter, sister–sister, and first‐cousin–first‐cousin dyads. After variable winnowing, inter‐individual distances or similarities were calculated using 11 odontometric variables and 25 dental morphological variables. Resulting distance matrices were ordinated in two dimensions using multidimensional scaling. Odontometric data performed relatively well at identifying known relative pairs, but the results were heavily affected by choice of similarity measure (e.g. Euclidean distances vs. Gower coefficients) and pre‐analysis data treatments (e.g. raw data vs. principal components). Dental morphological data performed comparably with odontometric data but were slightly less effective. Rare traits were identified and compared among relative pairs for concordance, with mixed results. Rare morphological features were randomly distributed throughout the population and were not exclusively found in close kin. In combination, results indicated the sister–sister dyad was most consistently identified; however, in no analysis were relatives more phenotypically similar than all random pairs of unrelated individuals. A multivariate, distance‐based approach was more effective than rare traits at identifying relative pairs, but even under ideal circumstances, there is not enough variation present in the dentition to faithfully identify close relatives in the absence of contextual archaeological data. Copyright © 2017 John Wiley & Sons, Ltd.