Approximately 40% of mobile phone use studies published in scholarly communication journals base their findings on self‐report data about how frequently respondents use their mobile phones. Using a subset of a larger representative sample we examine the validity of this type of self‐report data by comparing it to server log data. The self‐report data correlate only moderately with the server log data, indicating low criterion validity. The categorical self‐report measure asking respondents to estimate “how often” they use their mobile phones fared better than the continuous self‐report measure asking them to estimate their mobile phone activity “yesterday.” A multivariate exploratory analysis further suggests that it may be difficult to identify under‐ and overreporting using demographic variables alone.