Objectives The dementia diagnosis gap in sub‐Saharan Africa (SSA) is large, partly because of difficulties in screening for cognitive impairment in the community. As part of the Identification and Intervention for Dementia in Elderly Africans (IDEA) study, we aimed to validate the IDEA cognitive screen in a community‐based sample in rural Tanzania Methods Study participants were recruited from people who attended screening days held in villages within the rural Hai district of Tanzania. Criterion validity was assessed against the gold standard clinical dementia diagnosis using DSM‐IV criteria. Construct validity was assessed against, age, education, sex and grip strength and instrumental activities of daily living (IADLs). Internal consistency and floor and ceiling effects were also examined. Results During community screening, the IDEA cognitive screen had high criterion validity, with an area under the receiver operating characteristic curve of 0.855 (95% CI 0.794 to 0.915). Higher scores on the screen were significantly correlated with lower age, male sex, having attended school, better grip strength and improved performance in activities of daily living. Factor analysis revealed a single factor with an eigenvalue greater than one, although internal consistency was only moderate (Cronbach's alpha = 0.534). Conclusions The IDEA cognitive screen had high criterion and construct validity and is suitable for use as a cognitive screening instrument in a community setting in SSA. Only moderate internal consistency may partly reflect the multi‐domain nature of dementia as diagnosed clinically. Copyright © 2016 John Wiley & Sons, Ltd.