Many studies quantifying individual risk preferences of test persons show that results of different measuring methods may vary. Additional reservations about the reliability of the results regarding the risk attitude measurement arise from the fact that most studies are based on convenience groups, such as students or businessmen in developing countries. With this in mind, we systematically compare different measuring methods to answer the question how the choice of method affects the results. Moreover, we compare the risk preferences of German farmers with those of students and Kazakhstani farmers to investigate whether farmers’ risk preferences can be approximated through those of convenience groups. The methods applied comprise an incentive‐compatible Holt‐and‐Laury‐lottery as well as two psychometric methods. Results show that students respond consistently across all three elicitation methods whereas German and Kazakhstani farmers are more inconsistent. Significant differences exist in the responses of German students and German farmers. The comparison of risk preferences between German and Kazakhstani farmers, however, reveals significant similarities with respect to the psychometric methods.