Objective The advent of long‐term remotely conducted clinical trials requires assessments which can be administered online. This paper considers the utility, reliability, sensitivity and validity of an internet‐based system for measuring changes in cognitive function which is being used in one such trial. Methods The Platform for Research Online to investigate Genetics and Cognition in Ageing is a 10‐year longitudinal and entirely remote study launched in November 2015. The CogTrackTM System is being used to monitor changes in important aspects of cognitive function using tests of attention, information processing and episodic memory. On study entry, the participants performed CogTrackTM up to three times over seven days, and these data are evaluated in this paper. Results During the first six months of the study, 14 531 individuals aged 50 to 94 years enrolled and performed the CogTrackTM System, 8627 of whom completed three test sessions. On the first administration, 99.4% of the study tasks were successfully completed. Repeated testing showed training/familiarisation effects on four of the ten measures which had largely stabilised by the third test session. The factor structure of the various measures was found to be robust. Evaluation of the influence of age identified clinically relevant declines over the age range of the population on one or more measures from all tasks. Conclusions The results of these analyses identify CogTrackTM to be a practical and valid method to reliably, sensitively, remotely and repeatedly collect cognitive data from large samples of individuals aged 50 and over. Copyright © 2017 John Wiley & Sons, Ltd.