Though addressing sources of uncertainty is an important part of doing science, it has largely been neglected in assessing students' scientific argumentation. In this study, we initially defined a scientific argumentation construct in four structural elements consisting of claim, justification, uncertainty qualifier, and uncertainty rationale. We consulted literature to characterize and score different levels of student performances on each of these four argumentation elements. We designed a test comprised of nine scientific argumentation tasks addressing climate change, the search for life in space, and fresh water availability and administered it to 473 students from 9 high schools in the United States. After testing the local dependence and unidimensionality assumptions, we found that the uncertainty qualifier element was not aligned with the other three. After removing items related to uncertainty qualifier, we applied a Rasch analysis based on a Partial Credit Model. Results indicate that (1) claim, justification, and uncertainty rationale items form a unidimensional scale, (2) justification and uncertainty rationale items contribute the most on the unidimensional scientific argumentation scale as they cover much wider ranges of the scale than claim items, (3) average item difficulties increase in the order of claim, justification, and uncertainty rationale, (4) students' elaboration of uncertainty exhibits dual characteristics: self‐assessment of their own knowledge and ability versus scientific assessment of conceptual and empirical errors embedded in investigations, and (5) students who can make warrants between theory and evidence are more likely to think about uncertainty from scientific sources than those who cannot. We identified limitations of this study in terms of science topic coverage and sample selection and made suggestions on how these limitations might have affected results and interpretations. © 2014 Wiley Periodicals, Inc. J Res Sci Teach 51: 581–605, 2014