Given the centrality of argumentation in the Next Generation Science Standards, there is an urgent need for an empirically validated learning progression of this core practice and the development of high‐quality assessment items. Here, we introduce a hypothesized three‐tiered learning progression for scientific argumentation. The learning progression accounts for the intrinsic cognitive load associated with orchestrating arguments of increasingly complex structure. Our proposed learning progression for argumentation in science also makes an important distinction between construction and critique. We present validity evidence for this learning progression based on item response theory, and discuss the development of items used to test this learning progression. By analyzing data from cognitive think‐aloud interviews of students, written responses on pilot test administrations, and large‐scale test administrations using a Rasch analysis, we discuss the refinement both of our items and our learning progression to improve construct validity and scoring reliability. Limitations to this research as well as implications for future work on assessment of scientific argumentation are discussed. © 2016 Wiley Periodicals, Inc. J Res Sci Teach 53: 821–846, 2016