Aim To analyze whether sex hormone levels predict the incidence of type2 diabetes among elderly Finnish men. Methods This was a prospective population‐based study, with a 9‐year follow up period. The study population in the municipality of Lieto, Finland, consisted of elderly (age ≥64 years) men free of type 2 diabetes at baseline in 1998–1999 (n = 430). Body mass index and cardiovascular disease‐adjusted hazard ratios and their 95% confidence intervals for type 2 diabetes predicted by testosterone, free testosterone, sex hormone‐binding globulin, luteinizing hormone, and testosterone/luteinizing hormone were estimated. Results A total of 30 new cases of type 2 diabetes developed during the follow‐up period. After adjustment, only higher levels of testosterone (hazard ratio for one‐unit increase 0.93, 95% confidence interval 0.87–0.99, P = 0.020) and free testosterone (hazard ratio for 10‐unit increase 0.96, 95% confidence interval 0.91–1.00, P = 0.044) were associated with a lower risk of incident type 2 diabetes during the follow up. These associations (0.94, 95% confidence interval 0.87–1.00, P = 0.050 and 0.95, 95% confidence interval 0.90–1.00, P = 0.035, respectively) persisted even after additional adjustment of sex hormone‐binding globulin. Conclusion Higher levels of testosterone and free testosterone independently predicted a reduced risk of type 2 diabetes in the elderly men. Geriatr Gerontol Int 2014; ●●: ●●–●●.