Objective This study examined cancer knowledge in adolescent and young adult (AYA) survivors and pilot tested a Web‐based resource to provide individually tailored information regarding cancer treatment history, late effects risk, and resources. Methods Fifty‐two survivors (15–28 years old) who completed cancer treatment were recruited from the University of Minnesota oncology clinics. Participants were randomly assigned to receive access to personalized health history, late effects information, and resources via a password‐protected Web portal or to standard of care (physician counseling) only. Participants completed surveys measuring cancer knowledge, health locus of control, and psychosocial well‐being prior to randomization and approximately 1 year later. Results Overall, few participants accurately reported their chemotherapy history with detail (19% at baseline and 33% at follow‐up), and many did not recognize that previous cancer treatments could impact future health (60% at baseline and 54% at follow‐up). Among those randomized to the receive access to the website, utilization was very low, making it difficult to draw conclusions about efficacy. Nonetheless, these data suggest that offering tailored information through the Web was not more effective than standard of care at improving cancer knowledge. Anxiety and health beliefs were associated with cancer knowledge, including knowledge of steps survivors could take to mitigate late effects risks (p < 01). Conclusions Knowledge gaps exist among AYA survivors regarding important aspects of their treatment histories and ongoing health risks. Offering purely educational information (either in person by providers or via the Web) does not appear to be enough to close this gap. Copyright © 2015 John Wiley & Sons, Ltd.