To date no reliable and objective, change sensitive instrument for autistic symptoms is available. The brief observation of social communication change (BOSCC) was specifically developed to measure change of core autistic symptoms, for example, for use as outcome measure in early intervention trials. This study investigated quality criteria of a preliminary research version of the BOSCC in N = 21 children with autism spectrum disorder (ASD) who had participated for 1 year in the Frankfurt early intervention program (FFIP). BOSCC rating was done on play based ADOS video scenes. Inter‐rater agreement on the BOSCC average total was very high. The BOSCC showed a significant decrease of autistic symptoms after 1 year with a medium effect size. Symptom specific improvements were captured by the social communication subscale and most single items. The BOSCC showed comparable change sensitivity to other autism specific instruments. Future studies should focus on the finalized BOSCC version, and replicate findings in a larger sample. Autism Res 2016, 9: 940–950. © 2015 International Society for Autism Research, Wiley Periodicals, Inc.