The present study describes the development and validation of a video‐based situational judgment test (SJT) assessing social competencies in applicants to medical school. Study 1 explored the psychometric properties of the SJT based on two applicant cohorts (N1 = 769, N2 = 787). Study 2 compared SJT data from applicants and 90 medical students. Tests for mean differences, correlation and factor analyses were used. In Study 1, the supposed two‐factor model for the SJT must be rejected. According to the minimum average partial test a single factor solution explaining 19–21% of the variance was suggested. The SJT showed satisfactory psychometric properties and stronger correlations to personality than to cognition as well as high acceptance. In study 2, applicants reached higher SJT scores than students.