High scientific literacy is widely considered a public good. Methods of assessing public scientific knowledge or literacy are equally important. In an effort to measure lay scientific literacy in the United States, the National Science Foundation (NSF) science literacy scale has been a part of the last three waves of the General Social Survey. However, there has been debate over the validity of some survey items as indicators of science knowledge. While many researchers treat the NSF science scale as measuring a single dimension, previous work (Bann and Schwerin, 2004; Miller, 1998, 2004) suggests a bidimensional structure. This paper hypothesizes and tests a new measurement model for the NSF science knowledge scale and finds that two items about evolution and the big bang are more measures of a religious belief dimension termed "Young Earth Worldview" than they are measures of scientific knowledge. Results are replicated in seven samples.