Background Structured risk assessment is commonly used in secure settings to aid prediction and prevention of risky behaviours; the expected benefits have rarely been investigated. Aims The aim of the study is to determine whether adverse outcomes (physical and verbal aggression, self‐harm, victimisation, self‐neglect, unauthorised leave and substance abuse) reduced after patients were assessed with the Short‐term Assessment of Risk and Treatability (START). Methods In a naturalistic study, anonymised demographic and clinical information was collected from 50 male patients. Data included START assessment and frequency of target behaviour for 3 months before and after first assessment. Chi‐square and linear mixed models analyses were used to determine whether there was any change in the behaviours of interest. Results There were no significant changes in physical or verbal aggression over time, although a tendency towards fewer incidents was apparent. Other adverse behaviours were very infrequent at baseline, precluding adequate analysis. Conclusions In this small sample, START did not achieve its primary purpose of significant reduction in adverse behaviours. Although our sample size was informed by a power calculation, we may have over‐estimated the size of anticipated change. Further, the 3‐month comparison periods before and after the assessment follow‐up period were rather short. Accordingly, we recommend more research on the value of this tool in practice rather than discontinuing its use. Copyright © 2017 John Wiley & Sons, Ltd.