The authors examined the implications of test differences for defining and diagnosing comprehension deficits using reading comprehension tests. They had 995 children complete the Gray Oral Reading Test–3, the Qualitative Reading Inventory–3, the Woodcock–Johnson Passage Comprehension–3, and the Peabody Individual Achievement Test and compared which children were identified by each test as being in the lowest 10%. Although a child who performs so poorly might be expected to do poorly on all tests, the authors found that the average overlap between tests in diagnosing comprehension difficulties was only 43%. Consistency in diagnosis was greater for younger children, when comprehension deficits are the result of weaker decoding skills, than for older children. Inconsistencies between tests were just as evident when identifying the top performers. The different children identified as having a comprehension deficit by each test were compared on four profile variables—word decoding skill, IQ, ADHD symptoms, and working memory skill—to understand the nature of the different deficits assessed by each test. Theoretical and practical implications of these test differences in defining and diagnosing comprehension deficits are discussed.