Collaborative problem solving (CPS) assessment is a new academic research field with a number of educational implications. In 2015, the Programme for International Student Assessment (PISA) assessed CPS with a computer‐simulated human‐agent (H‐A) approach that claimed to measure 12 individual CPS skills for the first time. After reviewing the approach, we conceptually embedded a computer‐based collaborative behavior assessment (COLBAS) into the overarching PISA 2015 CPS approach. COLBAS is an H‐A CPS assessment instrument that can be used to measure certain aspects of CPS. In addition, we applied a model‐based cluster analysis to the embedded COLBAS aspects. The analysis revealed three types of student collaborator profiles that differed in cognitive performance and motivation: (a) passive low‐performing (non‐)collaborators, (b) active high‐performing collaborators, and (c) compensating collaborators.