This paper extends systematic analyses of maritime piracy by verifying the robustness of empirical results and examining the forecasting ability of empirical models. Recent research by Ward, Greenhill and Bakke () finds that statistically significant relationships frequently offer poor guidance when it comes to anticipating the inception of civil war. We assess the predictive ability of purported causal factors of piracy using evaluative statistical tools such as receiver‐operating characteristic plots, out‐of‐sample predictions, and outlier analysis. Statistical results for in‐sample and out‐of‐sample tests show that while factors such as military capacity, population size, coastline length, and trade volumes are statistically related to piracy, state fragility has by far the strongest predictive effect despite only being moderately statistically significant in the models. Outlier analysis demonstrates that while several countries experience higher numbers of piracy incidents than predicted, empirical models are generally robust to the presence of outliers. For policymakers, the findings suggest that counter‐piracy efforts focused on capacity‐building measures have the greatest potential for reducing the piracy threat.