Background Computer‐administered assessment of cognitive function is being increasingly incorporated in clinical trials; however, its performance in these settings has not been systematically evaluated. Design The Seniors Health and Activity Research Program pilot trial (N = 73) developed a computer‐based tool for assessing memory performance and executive functioning. The Lifestyle Interventions and Independence for Elders investigators incorporated this battery in a full‐scale multicenter clinical trial (N = 1635). We describe relationships that test scores have with those from interviewer‐administered cognitive function tests and risk factors for cognitive deficits and describe performance measures (completeness, intraclass correlations [ICC]). Results Computer‐based assessments of cognitive function had consistent relationships across the pilot and full‐scale trial cohorts with interviewer‐administered assessments of cognitive function, age, and a measure of physical function. In the Lifestyle Interventions and Independence for Elders cohort, their external validity was further demonstrated by associations with other risk factors for cognitive dysfunction: education, hypertension, diabetes, and physical function. Acceptable levels of data completeness (>83%) were achieved on all computer‐based measures; however, rates of missing data were higher among older participants (odds ratio = 1.06 for each additional year; p < 0.001) and those who reported no current computer use (odds ratio = 2.71; p < 0.001). ICCs among clinics were at least as low (ICC < 0.013) as for interviewer measures (ICC < 0.023), reflecting good standardization. All cognitive measures loaded onto the first principal component (global cognitive function), which accounted for 40% of the overall variance. Conclusion Our results support the use of computer‐based tools for assessing cognitive function in multicenter clinical trials of older individuals. Copyright © 2013 John Wiley & Sons, Ltd.