Continuous Performance Tests (CPTs) are used in research and clinical contexts to measure sustained attention and response inhibition. Reliability and validity of a new Online Continuous Performance Test (OCPT) was assessed. The OCPT is designed for delivery over the Internet, thereby opening new opportunities for research and clinical application in naturalistic settings. In Study 1, participants completed the OCPT twice over a 1-week period. One test was taken at home and one in the laboratory. Construct validity was assessed against a gold standard CPT measure. Results indicate acceptable reliability between the home- and laboratory-administered tests. Modest to high correlations were observed between the OCPT scales and the corresponding scales of the gold standard CPT. Study 2 examined whether the OCPT may discriminate participants with attention deficit hyperactivity disorder from healthy controls. Results revealed significantly higher rates of omission and commission errors and greater response time variability in participants with attention deficit hyperactivity disorder relative to healthy controls. These results support the reliability and validity of the OCPT and suggest that it may serve as an effective tool for the assessment of attention function in naturalistic settings.