Recent studies of authorship attribution have used machine‐learning methods including regularized multinomial logistic regression, neural nets, support vector machines, and the nearest shrunken centroid classifier to identify likely authors of disputed texts. These methods are all limited by an inability to perform open‐set classification and account for text and corpus size. We propose a customized Bayesian logit‐normal‐beta‐binomial classification model for supervised authorship attribution. The model is based on the beta‐binomial distribution with an explicit inverse relationship between extra‐binomial variation and text size. The model internally estimates the relationship of extra‐binomial variation to text size, and uses Markov Chain Monte Carlo (MCMC) to produce distributions of posterior authorship probabilities instead of point estimates. We illustrate the method by training the machine‐learning methods as well as the open‐set Bayesian classifier on undisputed papers of The Federalist, and testing the method on documents historically attributed to Alexander Hamilton, John Jay, and James Madison. The Bayesian classifier was the best classifier of these texts.