The portfolio optimization problem is investigated using a multivariate stochastic volatility model with factor dynamics, fat‐tailed errors and leverage effects. The efficient Markov chain Monte Carlo method is used to estimate model parameters, and the Rao–Blackwellized auxiliary particle filter is used to compute the likelihood and to predict conditional means and covariances. The proposed models are applied to sector indices of the Tokyo Stock Price Index (TOPIX), which consists of 33 stock market indices classified by industrial sectors. The portfolio is dynamically optimized under several expected utilities and two additional static strategies are considered as benchmarks. An extensive empirical study indicates that our proposed dynamic factor model with leverage or fat‐tailed errors significantly improves the predictions of the conditional mean and covariances, as well as various measures of portfolio performance.