Though unlikely virtues scales have a long history in personality, clinical, and applied psychology for detecting socially desirable responding, using such social desirability (SD) scales has generally failed to improve the validity of personality measures. We examined whether this is because (a) response distortion itself has minimal impact on personality's validity, (b) SD scales are ineffective at assessing response distortion, or (c) SD scales are conflated with substantive trait variance. We compiled a meta‐analytic multitrait multimethod matrix consisting of multirater personality traits, SD scales, and performance outcomes. We examined the influence of trait factors and self‐report method factors on SD scales and performance. We found that self‐report method variance (a) was negatively related to performance, (b) would suppress personality‐performance relationships for self‐report measures, and (c) was (partially) assessed by SD scales. However, relative to the effects of self‐report method variance, SD scales are even more strongly influenced by Conscientiousness, Emotional Stability, and Agreeableness. It is not the case that SD scales are insensitive to inflated responding but that their susceptibility to personality trait variance likely outweighs their benefits. We discuss the implications of these results for using SD scales in research and practice.